get_slice() inconsistent in 0.4?

classic Classic list List threaded Threaded
3 messages Options
br
Reply | Threaded
Open this post in threaded view
|

get_slice() inconsistent in 0.4?

br
Hi,

I have been playing with get_slice() in cassandra 0.4, and I have noticed the following issue:

When the column name is a Long, multiple invocations of get_slice() returns different values (sometimes none at all).

After a bit of a digging, I noticed that package org.apache.cassandra.db.marshal.LongType.java deserializes longs with a little endian byte order. The default java byte ordering is big endian. This seems to impact the way SliceQueryFilter iterates over the columns. Once I changed LongType to use big endian byte order, the problem went away for me.

Was wondering if this was a known issue (or an issue at all - cropped up by how I am using it).

My setup is as below:
- Cassandra 0.4
- Running on Mac OS Leopard 10.5.8, JDK 1.6.0_13
- Storage config has one keyspace, with one (simple) column family :
    <ColumnFamily CompareWith="LongType" Name="revs"/>
- When inserting data, I convert the long (my column name) to byte array as such:
        ByteArrayOutputStream bos = null;
        DataOutputStream dos = null;
        try {
        bos = new ByteArrayOutputStream();
        dos = new DataOutputStream(bos);
        dos.writeLong(l);
        dos.flush();
        byte[] ret =  bos.toByteArray();
        return ret;
- When querying, I use the below: (I use thrift. The example uses reverse order, I tried both ways)
         ColumnParent column_parent = new ColumnParent("revs", null);
         SlicePredicate predicate = new SlicePredicate();
         predicate.setSlice_range(new SliceRange(CarmotUtil.getBytes(System.currentTimeMillis()), CarmotUtil.getBytes(0l), true, 1000));            
        List<ColumnOrSuperColumn> ret = cl.get_slice("Objects", id, column_parent, predicate, 1);

I would appreciate any feedback/insights on this. I can provide more details as needed.

 Thanks,
Bhaskar

Reply | Threaded
Open this post in threaded view
|

Re: get_slice() inconsistent in 0.4?

Jonathan Ellis-3
You must be using an old version (beta1 maybe?)  We switched LongType
to big-endian in https://issues.apache.org/jira/browse/CASSANDRA-384.

-Jonathan

On Tue, Sep 1, 2009 at 9:28 PM, Bhaskar Rathod<[hidden email]> wrote:

> Hi,
>
> I have been playing with get_slice() in cassandra 0.4, and I have noticed the following issue:
>
> When the column name is a Long, multiple invocations of get_slice() returns different values (sometimes none at all).
>
> After a bit of a digging, I noticed that package org.apache.cassandra.db.marshal.LongType.java deserializes longs with a little endian byte order. The default java byte ordering is big endian. This seems to impact the way SliceQueryFilter iterates over the columns. Once I changed LongType to use big endian byte order, the problem went away for me.
>
> Was wondering if this was a known issue (or an issue at all - cropped up by how I am using it).
>
> My setup is as below:
> - Cassandra 0.4
> - Running on Mac OS Leopard 10.5.8, JDK 1.6.0_13
> - Storage config has one keyspace, with one (simple) column family :
>    <ColumnFamily CompareWith="LongType" Name="revs"/>
> - When inserting data, I convert the long (my column name) to byte array as such:
>        ByteArrayOutputStream bos = null;
>        DataOutputStream dos = null;
>        try {
>        bos = new ByteArrayOutputStream();
>        dos = new DataOutputStream(bos);
>        dos.writeLong(l);
>        dos.flush();
>        byte[] ret =  bos.toByteArray();
>        return ret;
> - When querying, I use the below: (I use thrift. The example uses reverse order, I tried both ways)
>         ColumnParent column_parent = new ColumnParent("revs", null);
>         SlicePredicate predicate = new SlicePredicate();
>         predicate.setSlice_range(new SliceRange(CarmotUtil.getBytes(System.currentTimeMillis()), CarmotUtil.getBytes(0l), true, 1000));
>        List<ColumnOrSuperColumn> ret = cl.get_slice("Objects", id, column_parent, predicate, 1);
>
> I would appreciate any feedback/insights on this. I can provide more details as needed.
>
>  Thanks,
> Bhaskar
>
>
br
Reply | Threaded
Open this post in threaded view
|

Re: get_slice() inconsistent in 0.4?

br
My version is definitely older than that. Will sync. Thanks.
( Apparently didn't look hard enough.)

Bhaskar




----- Original Message ----
From: Jonathan Ellis <[hidden email]>
To: [hidden email]
Sent: Tuesday, September 1, 2009 7:31:42 PM
Subject: Re: get_slice() inconsistent in 0.4?

You must be using an old version (beta1 maybe?)  We switched LongType
to big-endian in https://issues.apache.org/jira/browse/CASSANDRA-384.

-Jonathan

On Tue, Sep 1, 2009 at 9:28 PM, Bhaskar Rathod<[hidden email]> wrote:

> Hi,
>
> I have been playing with get_slice() in cassandra 0.4, and I have noticed the following issue:
>
> When the column name is a Long, multiple invocations of get_slice() returns different values (sometimes none at all).
>
> After a bit of a digging, I noticed that package org.apache.cassandra.db.marshal.LongType.java deserializes longs with a little endian byte order. The default java byte ordering is big endian. This seems to impact the way SliceQueryFilter iterates over the columns. Once I changed LongType to use big endian byte order, the problem went away for me.
>
> Was wondering if this was a known issue (or an issue at all - cropped up by how I am using it).
>
> My setup is as below:
> - Cassandra 0.4
> - Running on Mac OS Leopard 10.5.8, JDK 1.6.0_13
> - Storage config has one keyspace, with one (simple) column family :
>    <ColumnFamily CompareWith="LongType" Name="revs"/>
> - When inserting data, I convert the long (my column name) to byte array as such:
>        ByteArrayOutputStream bos = null;
>        DataOutputStream dos = null;
>        try {
>        bos = new ByteArrayOutputStream();
>        dos = new DataOutputStream(bos);
>        dos.writeLong(l);
>        dos.flush();
>        byte[] ret =  bos.toByteArray();
>        return ret;
> - When querying, I use the below: (I use thrift. The example uses reverse order, I tried both ways)
>         ColumnParent column_parent = new ColumnParent("revs", null);
>         SlicePredicate predicate = new SlicePredicate();
>         predicate.setSlice_range(new SliceRange(CarmotUtil.getBytes(System.currentTimeMillis()), CarmotUtil.getBytes(0l), true, 1000));
>        List<ColumnOrSuperColumn> ret = cl.get_slice("Objects", id, column_parent, predicate, 1);
>
> I would appreciate any feedback/insights on this. I can provide more details as needed.
>
>  Thanks,
> Bhaskar
>
>