Pls, help with fetching of super-column's value

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Pls, help with fetching of super-column's value

Teodor Sigaev
Hello!

I'm trying to implement schema of Twitter pointed in
http://blog.evanweaver.com/articles/2009/07/06/up-and-running-with-cassandra/ to
play and make experiments with Cassandra. That schema and code was developed for
Cassandra-0.3 and I'm trying to modify it to work with last Cassandra version.
I'm using Perl client generated by Thrift-r796538.

So, related fragments from storage-conf.xml:
         <Keyspace Name="Twitter">
             <KeysCachedFraction>0.01</KeysCachedFraction>
             <ColumnFamily CompareWith="AsciiType" Name="Statuses" />
             <ColumnFamily CompareWith="AsciiType" Name="Users" />
             <ColumnFamily ColumnType="Super" CompareWith="AsciiType"
CompareSubcolumnsWith="AsciiType" Name="UserRelationships" />
         </Keyspace>
Keyspace 'Twitter' is a second keyspace, first one is a default 'Keyspace1'

Fragments of code:
                 $result = $client->get_slice(
                     'Twitter',
                     $uid,
                     Cassandra::ColumnParent->new({
                         column_family=>'UserRelationships',
                         super_column=>'user_timeline',
                     }),
                     Cassandra::SlicePredicate->new({
                         #column_names=>['user_timeline'],
                         slice_range=>Cassandra::SliceRange->new({
                             reversed=>0,
                             count=>3
                         })
                     }),
                     Cassandra::ConsistencyLevel::ONE
                 );

It produces following error messages
$VAR1 = 'Can\'t use string ("0") as a SCALAR ref while "strict refs" in use at
/usr/lib/perl5/site_perl/5.8.8/Thrift/BinaryProtocol.pm line 376.'

If 'user_timeline' is moving to SlicePredicate (it's commented out in code
above) then get_slice returns all columns in super column. But I'd like to get
only 3 results...

With 0.3 version equivalent query works well:
         $result = $client->get_slice(
             'Twitter',
             $uid,
             'UserRelationships:user_timeline',
             0, 3
         );


What am I doing wrong? Thank you very much!

PS The same was in apache-cassandra-incubating-2009-08-17_13-23-14 snapshot too,
but I don't know about other versions/snapshot

--
Teodor Sigaev                                   E-mail: [hidden email]
                                                    WWW: http://www.sigaev.ru/
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Jonathan Ellis-3
2009/8/19 Teodor Sigaev <[hidden email]>:
> It produces following error messages
> $VAR1 = 'Can\'t use string ("0") as a SCALAR ref while "strict refs" in use
> at /usr/lib/perl5/site_perl/5.8.8/Thrift/BinaryProtocol.pm line 376.'

If it's objecting to the 0 in reversed=>0, it sounds like a perl
specific problem -- why would it be turning that into a string?

-Jonathan
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Evan Weaver
I just updated that blog post yesterday to use latest trunk.

I would say get the Ruby version working; then try to port it.

Evan

On Wed, Aug 19, 2009 at 9:48 AM, Jonathan Ellis<[hidden email]> wrote:

> 2009/8/19 Teodor Sigaev <[hidden email]>:
>> It produces following error messages
>> $VAR1 = 'Can\'t use string ("0") as a SCALAR ref while "strict refs" in use
>> at /usr/lib/perl5/site_perl/5.8.8/Thrift/BinaryProtocol.pm line 376.'
>
> If it's objecting to the 0 in reversed=>0, it sounds like a perl
> specific problem -- why would it be turning that into a string?
>
> -Jonathan
>



--
Evan Weaver
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Teodor Sigaev
In reply to this post by Jonathan Ellis-3
> If it's objecting to the 0 in reversed=>0, it sounds like a perl
> specific problem -- why would it be turning that into a string?
It seems to me that it isn't connected to reversed. I changed to 1 and nothing
was changed.

--
Teodor Sigaev                                   E-mail: [hidden email]
                                                    WWW: http://www.sigaev.ru/
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Evan Weaver
In the Ruby Thrift it expects a boolean, FYI. Maybe the "0 is false"
Perlism is shortcircuited?

2009/8/19 Teodor Sigaev <[hidden email]>:

>> If it's objecting to the 0 in reversed=>0, it sounds like a perl
>> specific problem -- why would it be turning that into a string?
>
> It seems to me that it isn't connected to reversed. I changed to 1 and
> nothing was changed.
>
> --
> Teodor Sigaev                                   E-mail: [hidden email]
>                                                   WWW: http://www.sigaev.ru/
>



--
Evan Weaver
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Jonathan Ellis-3
In reply to this post by Teodor Sigaev
2009/8/19 Teodor Sigaev <[hidden email]>:
>> If it's objecting to the 0 in reversed=>0, it sounds like a perl
>> specific problem -- why would it be turning that into a string?
>
> It seems to me that it isn't connected to reversed. I changed to 1 and
> nothing was changed.

Dunno, then.  Bug in the generated perl code?  Wouldn't be the first time.

-Jonathan
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Teodor Sigaev
> Dunno, then.  Bug in the generated perl code?  Wouldn't be the first time.

Interesting, if column_names is added with known status ids then Cassandra
returns them although it doesn't take into account reversed and count options.

                 $result = $client->get_slice(
                     'Twitter',
                     $uid,
                     Cassandra::ColumnParent->new({
                         column_family=>'UserRelationships',
                         super_column=>'user_timeline',
                     }),
                     Cassandra::SlicePredicate->new({
                         column_names=>\@StatusIds, #added line
                         slice_range=>Cassandra::SliceRange->new({
                             reversed=>0,
                             count=>3
                         })
                     }),
                     Cassandra::ConsistencyLevel::ONE


I install snapshot version of Thrift and regenerate Perl's interface - all is
the same.

--
Teodor Sigaev                                   E-mail: [hidden email]
                                                    WWW: http://www.sigaev.ru/
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Jonathan Ellis-3
2009/8/19 Teodor Sigaev <[hidden email]>:
>> Dunno, then.  Bug in the generated perl code?  Wouldn't be the first time.
>
> Interesting, if column_names is added with known status ids then Cassandra
> returns them although it doesn't take into account reversed and count
> options.

it's not supposed to.  column_names and slice_range are either/or.
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Teodor Sigaev
In reply to this post by Teodor Sigaev
Some more news, I added printing of stack trace to perl's client, and I see that
problem is in getting answer from server, not in sending. It breaks on reading
of exception (TMessageType::EXCEPTION) from server side.

Cassandra outputs (with loglevel DEBUG):
DEBUG - get_slice_from
ERROR - Internal error processing get_slice
java.lang.NullPointerException
         at java.lang.String.<init>(String.java:523)
         at org.apache.cassandra.db.marshal.AsciiType.getString(AsciiType.java:33)
         at
org.apache.cassandra.db.SliceFromReadCommand.toString(SliceFromReadCommand.java:71)
         at java.lang.String.valueOf(String.java:2838)
         at java.lang.StringBuilder.append(StringBuilder.java:132)
         at
org.apache.cassandra.service.StorageProxy.weakReadLocal(StorageProxy.java:602)
         at
org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:320)
         at
org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:92)
         at
org.apache.cassandra.service.CassandraServer.getSlice(CassandraServer.java:173)
         at
org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:213)
         at
org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:572)
         at
org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:560)
         at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
         at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
         at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
         at java.lang.Thread.run(Thread.java:636)


--
Teodor Sigaev                                   E-mail: [hidden email]
                                                    WWW: http://www.sigaev.ru/
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Michael Greene
start and finish in SliceRange are non-optional.  Try empty strings.

2009/8/19 Teodor Sigaev <[hidden email]>:

> Some more news, I added printing of stack trace to perl's client, and I see
> that problem is in getting answer from server, not in sending. It breaks on
> reading of exception (TMessageType::EXCEPTION) from server side.
>
> Cassandra outputs (with loglevel DEBUG):
> DEBUG - get_slice_from
> ERROR - Internal error processing get_slice
> java.lang.NullPointerException
>        at java.lang.String.<init>(String.java:523)
>        at
> org.apache.cassandra.db.marshal.AsciiType.getString(AsciiType.java:33)
>        at
> org.apache.cassandra.db.SliceFromReadCommand.toString(SliceFromReadCommand.java:71)
>        at java.lang.String.valueOf(String.java:2838)
>        at java.lang.StringBuilder.append(StringBuilder.java:132)
>        at
> org.apache.cassandra.service.StorageProxy.weakReadLocal(StorageProxy.java:602)
>        at
> org.apache.cassandra.service.StorageProxy.readProtocol(StorageProxy.java:320)
>        at
> org.apache.cassandra.service.CassandraServer.readColumnFamily(CassandraServer.java:92)
>        at
> org.apache.cassandra.service.CassandraServer.getSlice(CassandraServer.java:173)
>        at
> org.apache.cassandra.service.CassandraServer.get_slice(CassandraServer.java:213)
>        at
> org.apache.cassandra.service.Cassandra$Processor$get_slice.process(Cassandra.java:572)
>        at
> org.apache.cassandra.service.Cassandra$Processor.process(Cassandra.java:560)
>        at
> org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:252)
>        at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>        at java.lang.Thread.run(Thread.java:636)
>
>
> --
> Teodor Sigaev                                   E-mail: [hidden email]
>                                                   WWW: http://www.sigaev.ru/
>
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Teodor Sigaev
> start and finish in SliceRange are non-optional.  Try empty strings.

This is a partial fix :) - it works and doesn't emit any exception but returns
nothing.

--
Teodor Sigaev                                   E-mail: [hidden email]
                                                    WWW: http://www.sigaev.ru/
Reply | Threaded
Open this post in threaded view
|

Re: Pls, help with fetching of super-column's value

Drew Schleck
In reply to this post by Teodor Sigaev
Try setting start to "" and end to "~". This is what I did to "fix"
Lazyboy and it seems to work alright for now.

2009/8/19 Teodor Sigaev <[hidden email]>:
>> start and finish in SliceRange are non-optional.  Try empty strings.
>
> This is a partial fix :) - it works and doesn't emit any exception but
> returns nothing.
>
> --
> Teodor Sigaev                                   E-mail: [hidden email]
>                                                   WWW: http://www.sigaev.ru/
>