sstable writer and creating bytebuffers

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

sstable writer and creating bytebuffers

Peer, Oded

I am writing code to bulk load data into Cassandra using SSTableSimpleUnsortedWriter

I changed my partition key from a composite key (long, int) to a single column key (long).

For creating the composite key I used a CompositeType, and I kept using it after changing the key to a single column.

My code didn’t work until I changed the way I create the ByteBuffer not to use CompositeType.

 

The following code prints ‘false’.

Do you consider this a bug?

 

              long val = 123L;

              ByteBuffer direct = bytes( val );

              ByteBuffer composite = CompositeType.getInstance( LongType.instance ).builder().add( bytes( val ) ).build();

              System.out.println( direct.equals( composite ) );

 

Reply | Threaded
Open this post in threaded view
|

Re: sstable writer and creating bytebuffers

Sylvain Lebresne-3
No, it's not a bug. In a composite every elements start by a 2 short indicating the size of the element, plus an extra byte that is used for sorting purposes. A little bit more details can be found in the CompositeType class javadoc if you're interested. It's not the most compact format there is but changing it would break backward compatibility anyway. 

On Mon, Mar 30, 2015 at 12:38 PM, Peer, Oded <[hidden email]> wrote:

I am writing code to bulk load data into Cassandra using SSTableSimpleUnsortedWriter

I changed my partition key from a composite key (long, int) to a single column key (long).

For creating the composite key I used a CompositeType, and I kept using it after changing the key to a single column.

My code didn’t work until I changed the way I create the ByteBuffer not to use CompositeType.

 

The following code prints ‘false’.

Do you consider this a bug?

 

              long val = 123L;

              ByteBuffer direct = bytes( val );

              ByteBuffer composite = CompositeType.getInstance( LongType.instance ).builder().add( bytes( val ) ).build();

              System.out.println( direct.equals( composite ) );

 


Reply | Threaded
Open this post in threaded view
|

RE: sstable writer and creating bytebuffers

Peer, Oded

Thanks Sylvain.

Is there any way to create a composite key with only one column in Cassandra when creating a table, or should creating a CompositeType instance with a single column be prohibited?

 

 

From: Sylvain Lebresne [mailto:[hidden email]]
Sent: Monday, March 30, 2015 1:57 PM
To: [hidden email]
Subject: Re: sstable writer and creating bytebuffers

 

No, it's not a bug. In a composite every elements start by a 2 short indicating the size of the element, plus an extra byte that is used for sorting purposes. A little bit more details can be found in the CompositeType class javadoc if you're interested. It's not the most compact format there is but changing it would break backward compatibility anyway. 

 

On Mon, Mar 30, 2015 at 12:38 PM, Peer, Oded <[hidden email]> wrote:

I am writing code to bulk load data into Cassandra using SSTableSimpleUnsortedWriter

I changed my partition key from a composite key (long, int) to a single column key (long).

For creating the composite key I used a CompositeType, and I kept using it after changing the key to a single column.

My code didn’t work until I changed the way I create the ByteBuffer not to use CompositeType.

 

The following code prints ‘false’.

Do you consider this a bug?

 

              long val = 123L;

              ByteBuffer direct = bytes( val );

              ByteBuffer composite = CompositeType.getInstance( LongType.instance ).builder().add( bytes( val ) ).build();

              System.out.println( direct.equals( composite ) );

 

 

Reply | Threaded
Open this post in threaded view
|

Re: sstable writer and creating bytebuffers

Sylvain Lebresne-3
On Tue, Mar 31, 2015 at 7:42 AM, Peer, Oded <[hidden email]> wrote:

Thanks Sylvain.

Is there any way to create a composite key with only one column in Cassandra when creating a table, or should creating a CompositeType instance with a single column be prohibited?


It's hard to answer without knowing what you are trying to achieve. Provided I don't misunderstand what you are asking, then yes, it's technically possible, but it's hard to say how wise that is unless I know more about your constraints/the reasons you're considering that. Let's say that in general, if you have only a single column, then there isn't too much reasons to use a CompositeType.
 
--
Sylvain

 

 

From: Sylvain Lebresne [mailto:[hidden email]]
Sent: Monday, March 30, 2015 1:57 PM
To: [hidden email]
Subject: Re: sstable writer and creating bytebuffers

 

No, it's not a bug. In a composite every elements start by a 2 short indicating the size of the element, plus an extra byte that is used for sorting purposes. A little bit more details can be found in the CompositeType class javadoc if you're interested. It's not the most compact format there is but changing it would break backward compatibility anyway. 

 

On Mon, Mar 30, 2015 at 12:38 PM, Peer, Oded <[hidden email]> wrote:

I am writing code to bulk load data into Cassandra using SSTableSimpleUnsortedWriter

I changed my partition key from a composite key (long, int) to a single column key (long).

For creating the composite key I used a CompositeType, and I kept using it after changing the key to a single column.

My code didn’t work until I changed the way I create the ByteBuffer not to use CompositeType.

 

The following code prints ‘false’.

Do you consider this a bug?

 

              long val = 123L;

              ByteBuffer direct = bytes( val );

              ByteBuffer composite = CompositeType.getInstance( LongType.instance ).builder().add( bytes( val ) ).build();

              System.out.println( direct.equals( composite ) );

 

 


Reply | Threaded
Open this post in threaded view
|

RE: sstable writer and creating bytebuffers

Peer, Oded

I may have overcomplicated things.

In my opinion creating a CompositeType with a single type should throw an exception.

 

 

From: Sylvain Lebresne [mailto:[hidden email]]
Sent: Tuesday, March 31, 2015 10:18 AM
To: [hidden email]
Subject: Re: sstable writer and creating bytebuffers

 

On Tue, Mar 31, 2015 at 7:42 AM, Peer, Oded <[hidden email]> wrote:

Thanks Sylvain.

Is there any way to create a composite key with only one column in Cassandra when creating a table, or should creating a CompositeType instance with a single column be prohibited?

 

It's hard to answer without knowing what you are trying to achieve. Provided I don't misunderstand what you are asking, then yes, it's technically possible, but it's hard to say how wise that is unless I know more about your constraints/the reasons you're considering that. Let's say that in general, if you have only a single column, then there isn't too much reasons to use a CompositeType.

 
--

Sylvain

 

 

 

From: Sylvain Lebresne [mailto:[hidden email]]
Sent: Monday, March 30, 2015 1:57 PM
To: [hidden email]
Subject: Re: sstable writer and creating bytebuffers

 

No, it's not a bug. In a composite every elements start by a 2 short indicating the size of the element, plus an extra byte that is used for sorting purposes. A little bit more details can be found in the CompositeType class javadoc if you're interested. It's not the most compact format there is but changing it would break backward compatibility anyway. 

 

On Mon, Mar 30, 2015 at 12:38 PM, Peer, Oded <[hidden email]> wrote:

I am writing code to bulk load data into Cassandra using SSTableSimpleUnsortedWriter

I changed my partition key from a composite key (long, int) to a single column key (long).

For creating the composite key I used a CompositeType, and I kept using it after changing the key to a single column.

My code didn’t work until I changed the way I create the ByteBuffer not to use CompositeType.

 

The following code prints ‘false’.

Do you consider this a bug?

 

              long val = 123L;

              ByteBuffer direct = bytes( val );

              ByteBuffer composite = CompositeType.getInstance( LongType.instance ).builder().add( bytes( val ) ).build();

              System.out.println( direct.equals( composite ) );