CRUD test

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

CRUD test

Oleg Tsvinev
Hi there,

I'm trying to implement a simple CRUD service based on Cassandra. I use Hector client.

While writing tests, I found out that if I create a few columns using API, then delete them from cassandra-cli and and re-create them using the same code (same key, etc), I can never get  these new columns back using cassandra-cli. I tried to set different consistency levels but it did not change anything. I am never able to insert to these columns again from my code, although cassandra-cli can insert them.

I   thought it might have something to do with eventual consistency but even after waiting hours, nothing changes.

I have only one node (one cassandra sever) running on 64-bit Ubunutu, if it matters. I added my keyspace and couple of column families pretty much following defaults in storage-conf.xml.

Thanks,
  Oleg

Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Colin Vipurs
Have you checked the timestamp you're using for the subsequent inserts
is higher than that used in the delete?

On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev <[hidden email]> wrote:

> Hi there,
> I'm trying to implement a simple CRUD service based on Cassandra. I use
> Hector client.
> While writing tests, I found out that if I create a few columns using API,
> then delete them from cassandra-cli and and re-create them using the same
> code (same key, etc), I can never get  these new columns back using
> cassandra-cli. I tried to set different consistency levels but it did not
> change anything. I am never able to insert to these columns again from my
> code, although cassandra-cli can insert them.
> I   thought it might have something to do with eventual consistency but even
> after waiting hours, nothing changes.
> I have only one node (one cassandra sever) running on 64-bit Ubunutu, if it
> matters. I added my keyspace and couple of column families pretty much
> following defaults in storage-conf.xml.
> Thanks,
>   Oleg
>



--
Maybe she awoke to see the roommate's boyfriend swinging from the
chandelier wearing a boar's head.

Something which you, I, and everyone else would call "Tuesday", of course.
Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Oleg Tsvinev

Yes, and that was the issue. But now after I delete a row from cassandra-cli, I cannot insert anything back with my code. Insert code works does not throw any exceptions but when I read just inserted columns I get NotFoundException at the last line:

            client = borrowClient();
            Keyspace keyspace = client.getKeyspace(KEYSPACE, CONSISTENCY_LEVEL);
            ColumnPath cp = new ColumnPath(application);
            cp.setSuper_column(uuid.getBytes());
            SuperColumn sc = keyspace.getSuperColumn(category, cp);

It makes me think that once I remove supercolumn it cannot be created again.


On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:

Have you checked the timestamp you're using for the subsequent inserts
is higher than that used in the delete?


On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev <[hidden email]> wrote:
> Hi there,
> I'm try...

--
Maybe she awoke to see the roommate's boyfriend swinging from the
chandelier wearing a boar's head.

Something which you, I, and everyone else would call "Tuesday", of course.

Reply | Threaded
Open this post in threaded view
|

RE: CRUD test

Peter Minearo
I am able to reproduce his problem. If you take the default storage-conf.xml file and utilize the "Super2" ColumnFamily with the code below.  You will see that the data is not getting created once you run the delete.  It seems to not allow you to create data via Thrift.  HOWEVER, data can be created via the command line tool.

import java.io.UnsupportedEncodingException;
import java.util.List;

import org.apache.cassandra.thrift.Cassandra;
import org.apache.cassandra.thrift.Column;
import org.apache.cassandra.thrift.ColumnOrSuperColumn;
import org.apache.cassandra.thrift.ColumnParent;
import org.apache.cassandra.thrift.ColumnPath;
import org.apache.cassandra.thrift.ConsistencyLevel;
import org.apache.cassandra.thrift.InvalidRequestException;
import org.apache.cassandra.thrift.NotFoundException;
import org.apache.cassandra.thrift.SlicePredicate;
import org.apache.cassandra.thrift.SliceRange;
import org.apache.cassandra.thrift.SuperColumn;
import org.apache.cassandra.thrift.TimedOutException;
import org.apache.cassandra.thrift.UnavailableException;
import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;


public class CrudTest {

       
        private static final String KEYSPACE = "Keyspace1";


        public static void main(String[] args) {
                CrudTest client = new CrudTest();
               
                try {
                        client.run();
                } catch (Exception e) {
                        e.printStackTrace();
                }
               
        }
       
       
        public void run() throws TException, InvalidRequestException, UnavailableException, UnsupportedEncodingException, NotFoundException, TimedOutException {
                TTransport tr = new TSocket("localhost", 9160);
                TProtocol proto = new TBinaryProtocol(tr);
                Cassandra.Client client = new Cassandra.Client(proto);
                tr.open();
               
                System.out.println("******** CREATING DATA *********");
                createData(client);
                getData(client);
                System.out.println();
                System.out.println("******** DELETING DATA *********");
                deleteData(client);
                getData(client);
                System.out.println();
                System.out.println("******** CREATING DATA *********");
                createData(client);
                getData(client);
               
                tr.close();
          }

       
        private void createData(Cassandra.Client client) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
                ColumnPath cp1 = new ColumnPath("Super2");
                cp1.setSuper_column("hotel".getBytes());
                cp1.setColumn("Best Western".getBytes());
               
               
                client.insert(KEYSPACE,
                          "name",
                          cp1,
                          "Best Western of SF".getBytes(),
                          System.currentTimeMillis(),
                          ConsistencyLevel.ALL);

                ColumnPath cp2 = new ColumnPath("Super2");
                cp2.setSuper_column("hotel".getBytes());
                cp2.setColumn("Econolodge".getBytes());
               
                client.insert(KEYSPACE,
                                  "name",
                                  cp2,
                                  "Econolodge of SF".getBytes(),
                                  System.currentTimeMillis(),
                                  ConsistencyLevel.ALL);
               
        }

       
        private void deleteData(Cassandra.Client client) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
               
                client.remove(KEYSPACE,
                                  "hotel",
                                  new ColumnPath("Super2"),
                                  System.currentTimeMillis(),
                                  ConsistencyLevel.ONE);
               
        }
       
       
        private void getData(Cassandra.Client client) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
                SliceRange sliceRange = new SliceRange();
                sliceRange.setStart(new byte[] {});
                sliceRange.setFinish(new byte[] {});
               
                SlicePredicate slicePredicate = new SlicePredicate();
                slicePredicate.setSlice_range(sliceRange);
               
                getData(client, slicePredicate);
        }


        private void getData(Cassandra.Client client, SlicePredicate slicePredicate) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
                List<ColumnOrSuperColumn> coscList = client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"), slicePredicate, ConsistencyLevel.ALL);
               
                if (coscList.isEmpty()) {
                        System.out.println("Column Or Super Column is EMPTY");
                }
               
                for (ColumnOrSuperColumn cosc: coscList) {
                       
                        if (cosc == null) {
                                System.out.println("NULL RETURN VALUE");
                        }
                       
                        SuperColumn superColumn = cosc.getSuper_column();
                       
                        if (superColumn == null) {
                                System.out.println("Super Column is NULL");
                        }
                        else {
                                showSuperColumnInfo(superColumn);
                        }
                       
                }
               
        }


        private void showSuperColumnInfo(SuperColumn superColumn) {
                System.out.println("######## Super Columns ###########");
                System.out.println("Super Column Name = " + new String(superColumn.getName()));
                List<Column> columnList = superColumn.getColumns();
               
                System.out.println("--------- Start Columns -----------");
                for (Column column: columnList) {
                        System.out.println("Column Name = " + new String(column.getName()));
                        System.out.println("Column Value = " + new String(column.getValue()));
                }
                System.out.println("--------- End Columns -----------");
                System.out.println("##################################");
        }
       
       
}




-----Original Message-----
From: Oleg Tsvinev [mailto:[hidden email]]
Sent: Thu 7/22/2010 1:56 PM
To: [hidden email]
Subject: Re: CRUD test
 
Yes, and that was the issue. But now after I delete a row from
cassandra-cli, I cannot insert anything back with my code. Insert code works
does not throw any exceptions but when I read just inserted columns I get
NotFoundException at the last line:

            client = borrowClient();
            Keyspace keyspace = client.getKeyspace(KEYSPACE,
CONSISTENCY_LEVEL);
            ColumnPath cp = new ColumnPath(application);
            cp.setSuper_column(uuid.getBytes());
            SuperColumn sc = keyspace.getSuperColumn(category, cp);

It makes me think that once I remove supercolumn it cannot be created again.


On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:

Have you checked the timestamp you're using for the subsequent inserts
is higher than that used in the delete?


On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev <[hidden email]>
wrote:
> Hi there,
> I'm try...
--
Maybe she awoke to see the roommate's boyfriend swinging from the
chandelier wearing a boar's head.

Something which you, I, and everyone else would call "Tuesday", of course.


winmail.dat (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Jonathan Shook
I suspect that it is still your timestamps.
You can verify this with a fake timestamp generator that is simply
incremented on each getTimestamp().

1 millisecond is a long time for code that is wrapped tightly in a
test. You are likely using the same logical time stamp for multiple
operations.


On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
<[hidden email]> wrote:

> I am able to reproduce his problem. If you take the default storage-conf.xml file and utilize the "Super2" ColumnFamily with the code below.  You will see that the data is not getting created once you run the delete.  It seems to not allow you to create data via Thrift.  HOWEVER, data can be created via the command line tool.
>
> import java.io.UnsupportedEncodingException;
> import java.util.List;
>
> import org.apache.cassandra.thrift.Cassandra;
> import org.apache.cassandra.thrift.Column;
> import org.apache.cassandra.thrift.ColumnOrSuperColumn;
> import org.apache.cassandra.thrift.ColumnParent;
> import org.apache.cassandra.thrift.ColumnPath;
> import org.apache.cassandra.thrift.ConsistencyLevel;
> import org.apache.cassandra.thrift.InvalidRequestException;
> import org.apache.cassandra.thrift.NotFoundException;
> import org.apache.cassandra.thrift.SlicePredicate;
> import org.apache.cassandra.thrift.SliceRange;
> import org.apache.cassandra.thrift.SuperColumn;
> import org.apache.cassandra.thrift.TimedOutException;
> import org.apache.cassandra.thrift.UnavailableException;
> import org.apache.thrift.TException;
> import org.apache.thrift.protocol.TBinaryProtocol;
> import org.apache.thrift.protocol.TProtocol;
> import org.apache.thrift.transport.TSocket;
> import org.apache.thrift.transport.TTransport;
>
>
> public class CrudTest {
>
>
>        private static final String KEYSPACE = "Keyspace1";
>
>
>        public static void main(String[] args) {
>                CrudTest client = new CrudTest();
>
>                try {
>                        client.run();
>                } catch (Exception e) {
>                        e.printStackTrace();
>                }
>
>        }
>
>
>        public void run() throws TException, InvalidRequestException, UnavailableException, UnsupportedEncodingException, NotFoundException, TimedOutException {
>                TTransport tr = new TSocket("localhost", 9160);
>                TProtocol proto = new TBinaryProtocol(tr);
>                Cassandra.Client client = new Cassandra.Client(proto);
>                tr.open();
>
>                System.out.println("******** CREATING DATA *********");
>                createData(client);
>                getData(client);
>                System.out.println();
>                System.out.println("******** DELETING DATA *********");
>                deleteData(client);
>                getData(client);
>                System.out.println();
>                System.out.println("******** CREATING DATA *********");
>                createData(client);
>                getData(client);
>
>                tr.close();
>          }
>
>
>        private void createData(Cassandra.Client client) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
>                ColumnPath cp1 = new ColumnPath("Super2");
>                cp1.setSuper_column("hotel".getBytes());
>                cp1.setColumn("Best Western".getBytes());
>
>
>                client.insert(KEYSPACE,
>                          "name",
>                          cp1,
>                          "Best Western of SF".getBytes(),
>                          System.currentTimeMillis(),
>                          ConsistencyLevel.ALL);
>
>                ColumnPath cp2 = new ColumnPath("Super2");
>                cp2.setSuper_column("hotel".getBytes());
>                cp2.setColumn("Econolodge".getBytes());
>
>                client.insert(KEYSPACE,
>                                  "name",
>                                  cp2,
>                                  "Econolodge of SF".getBytes(),
>                                  System.currentTimeMillis(),
>                                  ConsistencyLevel.ALL);
>
>        }
>
>
>        private void deleteData(Cassandra.Client client) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
>
>                client.remove(KEYSPACE,
>                                  "hotel",
>                                  new ColumnPath("Super2"),
>                                  System.currentTimeMillis(),
>                                  ConsistencyLevel.ONE);
>
>        }
>
>
>        private void getData(Cassandra.Client client) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
>                SliceRange sliceRange = new SliceRange();
>                sliceRange.setStart(new byte[] {});
>                sliceRange.setFinish(new byte[] {});
>
>                SlicePredicate slicePredicate = new SlicePredicate();
>                slicePredicate.setSlice_range(sliceRange);
>
>                getData(client, slicePredicate);
>        }
>
>
>        private void getData(Cassandra.Client client, SlicePredicate slicePredicate) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
>                List<ColumnOrSuperColumn> coscList = client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"), slicePredicate, ConsistencyLevel.ALL);
>
>                if (coscList.isEmpty()) {
>                        System.out.println("Column Or Super Column is EMPTY");
>                }
>
>                for (ColumnOrSuperColumn cosc: coscList) {
>
>                        if (cosc == null) {
>                                System.out.println("NULL RETURN VALUE");
>                        }
>
>                        SuperColumn superColumn = cosc.getSuper_column();
>
>                        if (superColumn == null) {
>                                System.out.println("Super Column is NULL");
>                        }
>                        else {
>                                showSuperColumnInfo(superColumn);
>                        }
>
>                }
>
>        }
>
>
>        private void showSuperColumnInfo(SuperColumn superColumn) {
>                System.out.println("######## Super Columns ###########");
>                System.out.println("Super Column Name = " + new String(superColumn.getName()));
>                List<Column> columnList = superColumn.getColumns();
>
>                System.out.println("--------- Start Columns -----------");
>                for (Column column: columnList) {
>                        System.out.println("Column Name = " + new String(column.getName()));
>                        System.out.println("Column Value = " + new String(column.getValue()));
>                }
>                System.out.println("--------- End Columns -----------");
>                System.out.println("##################################");
>        }
>
>
> }
>
>
>
>
> -----Original Message-----
> From: Oleg Tsvinev [mailto:[hidden email]]
> Sent: Thu 7/22/2010 1:56 PM
> To: [hidden email]
> Subject: Re: CRUD test
>
> Yes, and that was the issue. But now after I delete a row from
> cassandra-cli, I cannot insert anything back with my code. Insert code works
> does not throw any exceptions but when I read just inserted columns I get
> NotFoundException at the last line:
>
>            client = borrowClient();
>            Keyspace keyspace = client.getKeyspace(KEYSPACE,
> CONSISTENCY_LEVEL);
>            ColumnPath cp = new ColumnPath(application);
>            cp.setSuper_column(uuid.getBytes());
>            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>
> It makes me think that once I remove supercolumn it cannot be created again.
>
>
> On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:
>
> Have you checked the timestamp you're using for the subsequent inserts
> is higher than that used in the delete?
>
>
> On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev <[hidden email]>
> wrote:
>> Hi there,
>> I'm try...
> --
> Maybe she awoke to see the roommate's boyfriend swinging from the
> chandelier wearing a boar's head.
>
> Something which you, I, and everyone else would call "Tuesday", of course.
>
>
Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Oleg Tsvinev
Johathan,

I followed your suggestion. Unfortunately, CRUD test still does not work for me. Can you provide a simplest CRUD test possible that works?

On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]> wrote:
I suspect that it is still your timestamps.
You can verify this with a fake timestamp generator that is simply
incremented on each getTimestamp().

1 millisecond is a long time for code that is wrapped tightly in a
test. You are likely using the same logical time stamp for multiple
operations.


On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
<[hidden email]> wrote:
> I am able to reproduce his problem. If you take the default storage-conf.xml file and utilize the "Super2" ColumnFamily with the code below.  You will see that the data is not getting created once you run the delete.  It seems to not allow you to create data via Thrift.  HOWEVER, data can be created via the command line tool.
>
> import java.io.UnsupportedEncodingException;
> import java.util.List;
>
> import org.apache.cassandra.thrift.Cassandra;
> import org.apache.cassandra.thrift.Column;
> import org.apache.cassandra.thrift.ColumnOrSuperColumn;
> import org.apache.cassandra.thrift.ColumnParent;
> import org.apache.cassandra.thrift.ColumnPath;
> import org.apache.cassandra.thrift.ConsistencyLevel;
> import org.apache.cassandra.thrift.InvalidRequestException;
> import org.apache.cassandra.thrift.NotFoundException;
> import org.apache.cassandra.thrift.SlicePredicate;
> import org.apache.cassandra.thrift.SliceRange;
> import org.apache.cassandra.thrift.SuperColumn;
> import org.apache.cassandra.thrift.TimedOutException;
> import org.apache.cassandra.thrift.UnavailableException;
> import org.apache.thrift.TException;
> import org.apache.thrift.protocol.TBinaryProtocol;
> import org.apache.thrift.protocol.TProtocol;
> import org.apache.thrift.transport.TSocket;
> import org.apache.thrift.transport.TTransport;
>
>
> public class CrudTest {
>
>
>        private static final String KEYSPACE = "Keyspace1";
>
>
>        public static void main(String[] args) {
>                CrudTest client = new CrudTest();
>
>                try {
>                        client.run();
>                } catch (Exception e) {
>                        e.printStackTrace();
>                }
>
>        }
>
>
>        public void run() throws TException, InvalidRequestException, UnavailableException, UnsupportedEncodingException, NotFoundException, TimedOutException {
>                TTransport tr = new TSocket("localhost", 9160);
>                TProtocol proto = new TBinaryProtocol(tr);
>                Cassandra.Client client = new Cassandra.Client(proto);
>                tr.open();
>
>                System.out.println("******** CREATING DATA *********");
>                createData(client);
>                getData(client);
>                System.out.println();
>                System.out.println("******** DELETING DATA *********");
>                deleteData(client);
>                getData(client);
>                System.out.println();
>                System.out.println("******** CREATING DATA *********");
>                createData(client);
>                getData(client);
>
>                tr.close();
>          }
>
>
>        private void createData(Cassandra.Client client) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
>                ColumnPath cp1 = new ColumnPath("Super2");
>                cp1.setSuper_column("hotel".getBytes());
>                cp1.setColumn("Best Western".getBytes());
>
>
>                client.insert(KEYSPACE,
>                          "name",
>                          cp1,
>                          "Best Western of SF".getBytes(),
>                          System.currentTimeMillis(),
>                          ConsistencyLevel.ALL);
>
>                ColumnPath cp2 = new ColumnPath("Super2");
>                cp2.setSuper_column("hotel".getBytes());
>                cp2.setColumn("Econolodge".getBytes());
>
>                client.insert(KEYSPACE,
>                                  "name",
>                                  cp2,
>                                  "Econolodge of SF".getBytes(),
>                                  System.currentTimeMillis(),
>                                  ConsistencyLevel.ALL);
>
>        }
>
>
>        private void deleteData(Cassandra.Client client) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
>
>                client.remove(KEYSPACE,
>                                  "hotel",
>                                  new ColumnPath("Super2"),
>                                  System.currentTimeMillis(),
>                                  ConsistencyLevel.ONE);
>
>        }
>
>
>        private void getData(Cassandra.Client client) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
>                SliceRange sliceRange = new SliceRange();
>                sliceRange.setStart(new byte[] {});
>                sliceRange.setFinish(new byte[] {});
>
>                SlicePredicate slicePredicate = new SlicePredicate();
>                slicePredicate.setSlice_range(sliceRange);
>
>                getData(client, slicePredicate);
>        }
>
>
>        private void getData(Cassandra.Client client, SlicePredicate slicePredicate) throws InvalidRequestException, UnavailableException, TimedOutException, TException {
>                List<ColumnOrSuperColumn> coscList = client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"), slicePredicate, ConsistencyLevel.ALL);
>
>                if (coscList.isEmpty()) {
>                        System.out.println("Column Or Super Column is EMPTY");
>                }
>
>                for (ColumnOrSuperColumn cosc: coscList) {
>
>                        if (cosc == null) {
>                                System.out.println("NULL RETURN VALUE");
>                        }
>
>                        SuperColumn superColumn = cosc.getSuper_column();
>
>                        if (superColumn == null) {
>                                System.out.println("Super Column is NULL");
>                        }
>                        else {
>                                showSuperColumnInfo(superColumn);
>                        }
>
>                }
>
>        }
>
>
>        private void showSuperColumnInfo(SuperColumn superColumn) {
>                System.out.println("######## Super Columns ###########");
>                System.out.println("Super Column Name = " + new String(superColumn.getName()));
>                List<Column> columnList = superColumn.getColumns();
>
>                System.out.println("--------- Start Columns -----------");
>                for (Column column: columnList) {
>                        System.out.println("Column Name = " + new String(column.getName()));
>                        System.out.println("Column Value = " + new String(column.getValue()));
>                }
>                System.out.println("--------- End Columns -----------");
>                System.out.println("##################################");
>        }
>
>
> }
>
>
>
>
> -----Original Message-----
> From: Oleg Tsvinev [mailto:[hidden email]]
> Sent: Thu 7/22/2010 1:56 PM
> To: [hidden email]
> Subject: Re: CRUD test
>
> Yes, and that was the issue. But now after I delete a row from
> cassandra-cli, I cannot insert anything back with my code. Insert code works
> does not throw any exceptions but when I read just inserted columns I get
> NotFoundException at the last line:
>
>            client = borrowClient();
>            Keyspace keyspace = client.getKeyspace(KEYSPACE,
> CONSISTENCY_LEVEL);
>            ColumnPath cp = new ColumnPath(application);
>            cp.setSuper_column(uuid.getBytes());
>            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>
> It makes me think that once I remove supercolumn it cannot be created again.
>
>
> On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:
>
> Have you checked the timestamp you're using for the subsequent inserts
> is higher than that used in the delete?
>
>
> On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev <[hidden email]>
> wrote:
>> Hi there,
>> I'm try...
> --
> Maybe she awoke to see the roommate's boyfriend swinging from the
> chandelier wearing a boar's head.
>
> Something which you, I, and everyone else would call "Tuesday", of course.
>
>

Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Jonathan Shook
There seem to be data consistency bugs in the test.  Are "name" and
"hotel" being used in a pair-wise way?
Specifically, the first test is using creating one and checking for the other.

On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]> wrote:

> Johathan,
> I followed your suggestion. Unfortunately, CRUD test still does not work for
> me. Can you provide a simplest CRUD test possible that works?
> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]> wrote:
>>
>> I suspect that it is still your timestamps.
>> You can verify this with a fake timestamp generator that is simply
>> incremented on each getTimestamp().
>>
>> 1 millisecond is a long time for code that is wrapped tightly in a
>> test. You are likely using the same logical time stamp for multiple
>> operations.
>>
>>
>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>> <[hidden email]> wrote:
>> > I am able to reproduce his problem. If you take the default
>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with the code
>> > below.  You will see that the data is not getting created once you run the
>> > delete.  It seems to not allow you to create data via Thrift.  HOWEVER, data
>> > can be created via the command line tool.
>> >
>> > import java.io.UnsupportedEncodingException;
>> > import java.util.List;
>> >
>> > import org.apache.cassandra.thrift.Cassandra;
>> > import org.apache.cassandra.thrift.Column;
>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>> > import org.apache.cassandra.thrift.ColumnParent;
>> > import org.apache.cassandra.thrift.ColumnPath;
>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>> > import org.apache.cassandra.thrift.InvalidRequestException;
>> > import org.apache.cassandra.thrift.NotFoundException;
>> > import org.apache.cassandra.thrift.SlicePredicate;
>> > import org.apache.cassandra.thrift.SliceRange;
>> > import org.apache.cassandra.thrift.SuperColumn;
>> > import org.apache.cassandra.thrift.TimedOutException;
>> > import org.apache.cassandra.thrift.UnavailableException;
>> > import org.apache.thrift.TException;
>> > import org.apache.thrift.protocol.TBinaryProtocol;
>> > import org.apache.thrift.protocol.TProtocol;
>> > import org.apache.thrift.transport.TSocket;
>> > import org.apache.thrift.transport.TTransport;
>> >
>> >
>> > public class CrudTest {
>> >
>> >
>> >        private static final String KEYSPACE = "Keyspace1";
>> >
>> >
>> >        public static void main(String[] args) {
>> >                CrudTest client = new CrudTest();
>> >
>> >                try {
>> >                        client.run();
>> >                } catch (Exception e) {
>> >                        e.printStackTrace();
>> >                }
>> >
>> >        }
>> >
>> >
>> >        public void run() throws TException, InvalidRequestException,
>> > UnavailableException, UnsupportedEncodingException, NotFoundException,
>> > TimedOutException {
>> >                TTransport tr = new TSocket("localhost", 9160);
>> >                TProtocol proto = new TBinaryProtocol(tr);
>> >                Cassandra.Client client = new Cassandra.Client(proto);
>> >                tr.open();
>> >
>> >                System.out.println("******** CREATING DATA *********");
>> >                createData(client);
>> >                getData(client);
>> >                System.out.println();
>> >                System.out.println("******** DELETING DATA *********");
>> >                deleteData(client);
>> >                getData(client);
>> >                System.out.println();
>> >                System.out.println("******** CREATING DATA *********");
>> >                createData(client);
>> >                getData(client);
>> >
>> >                tr.close();
>> >          }
>> >
>> >
>> >        private void createData(Cassandra.Client client) throws
>> > InvalidRequestException, UnavailableException, TimedOutException, TException
>> > {
>> >                ColumnPath cp1 = new ColumnPath("Super2");
>> >                cp1.setSuper_column("hotel".getBytes());
>> >                cp1.setColumn("Best Western".getBytes());
>> >
>> >
>> >                client.insert(KEYSPACE,
>> >                          "name",
>> >                          cp1,
>> >                          "Best Western of SF".getBytes(),
>> >                          System.currentTimeMillis(),
>> >                          ConsistencyLevel.ALL);
>> >
>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >                cp2.setSuper_column("hotel".getBytes());
>> >                cp2.setColumn("Econolodge".getBytes());
>> >
>> >                client.insert(KEYSPACE,
>> >                                  "name",
>> >                                  cp2,
>> >                                  "Econolodge of SF".getBytes(),
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ALL);
>> >
>> >        }
>> >
>> >
>> >        private void deleteData(Cassandra.Client client) throws
>> > InvalidRequestException, UnavailableException, TimedOutException, TException
>> > {
>> >
>> >                client.remove(KEYSPACE,
>> >                                  "hotel",
>> >                                  new ColumnPath("Super2"),
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ONE);
>> >
>> >        }
>> >
>> >
>> >        private void getData(Cassandra.Client client) throws
>> > InvalidRequestException, UnavailableException, TimedOutException, TException
>> > {
>> >                SliceRange sliceRange = new SliceRange();
>> >                sliceRange.setStart(new byte[] {});
>> >                sliceRange.setFinish(new byte[] {});
>> >
>> >                SlicePredicate slicePredicate = new SlicePredicate();
>> >                slicePredicate.setSlice_range(sliceRange);
>> >
>> >                getData(client, slicePredicate);
>> >        }
>> >
>> >
>> >        private void getData(Cassandra.Client client, SlicePredicate
>> > slicePredicate) throws InvalidRequestException, UnavailableException,
>> > TimedOutException, TException {
>> >                List<ColumnOrSuperColumn> coscList =
>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>> > slicePredicate, ConsistencyLevel.ALL);
>> >
>> >                if (coscList.isEmpty()) {
>> >                        System.out.println("Column Or Super Column is
>> > EMPTY");
>> >                }
>> >
>> >                for (ColumnOrSuperColumn cosc: coscList) {
>> >
>> >                        if (cosc == null) {
>> >                                System.out.println("NULL RETURN VALUE");
>> >                        }
>> >
>> >                        SuperColumn superColumn = cosc.getSuper_column();
>> >
>> >                        if (superColumn == null) {
>> >                                System.out.println("Super Column is
>> > NULL");
>> >                        }
>> >                        else {
>> >                                showSuperColumnInfo(superColumn);
>> >                        }
>> >
>> >                }
>> >
>> >        }
>> >
>> >
>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>> >                System.out.println("######## Super Columns ###########");
>> >                System.out.println("Super Column Name = " + new
>> > String(superColumn.getName()));
>> >                List<Column> columnList = superColumn.getColumns();
>> >
>> >                System.out.println("--------- Start Columns
>> > -----------");
>> >                for (Column column: columnList) {
>> >                        System.out.println("Column Name = " + new
>> > String(column.getName()));
>> >                        System.out.println("Column Value = " + new
>> > String(column.getValue()));
>> >                }
>> >                System.out.println("--------- End Columns -----------");
>> >                System.out.println("##################################");
>> >        }
>> >
>> >
>> > }
>> >
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Oleg Tsvinev [mailto:[hidden email]]
>> > Sent: Thu 7/22/2010 1:56 PM
>> > To: [hidden email]
>> > Subject: Re: CRUD test
>> >
>> > Yes, and that was the issue. But now after I delete a row from
>> > cassandra-cli, I cannot insert anything back with my code. Insert code
>> > works
>> > does not throw any exceptions but when I read just inserted columns I
>> > get
>> > NotFoundException at the last line:
>> >
>> >            client = borrowClient();
>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>> > CONSISTENCY_LEVEL);
>> >            ColumnPath cp = new ColumnPath(application);
>> >            cp.setSuper_column(uuid.getBytes());
>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>> >
>> > It makes me think that once I remove supercolumn it cannot be created
>> > again.
>> >
>> >
>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:
>> >
>> > Have you checked the timestamp you're using for the subsequent inserts
>> > is higher than that used in the delete?
>> >
>> >
>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev <[hidden email]>
>> > wrote:
>> >> Hi there,
>> >> I'm try...
>> > --
>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>> > chandelier wearing a boar's head.
>> >
>> > Something which you, I, and everyone else would call "Tuesday", of
>> > course.
>> >
>> >
>
>
Reply | Threaded
Open this post in threaded view
|

RE: CRUD test

Peter Minearo
The Model Should look like:


Super2 = {
        hotel: {
                        Best Western: {name: "Best Western of SF"}
                        Econolodge: {name: "Econolodge of SF"}
        }
}

Are the CRUD Operations not referencing this correctly?



-----Original Message-----
From: Jonathan Shook [mailto:[hidden email]]
Sent: Friday, July 23, 2010 1:34 PM
To: [hidden email]
Subject: Re: CRUD test

There seem to be data consistency bugs in the test.  Are "name" and "hotel" being used in a pair-wise way?
Specifically, the first test is using creating one and checking for the other.

On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]> wrote:

> Johathan,
> I followed your suggestion. Unfortunately, CRUD test still does not
> work for me. Can you provide a simplest CRUD test possible that works?
> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]> wrote:
>>
>> I suspect that it is still your timestamps.
>> You can verify this with a fake timestamp generator that is simply
>> incremented on each getTimestamp().
>>
>> 1 millisecond is a long time for code that is wrapped tightly in a
>> test. You are likely using the same logical time stamp for multiple
>> operations.
>>
>>
>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>> <[hidden email]> wrote:
>> > I am able to reproduce his problem. If you take the default
>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>> > the code below.  You will see that the data is not getting created
>> > once you run the delete.  It seems to not allow you to create data
>> > via Thrift.  HOWEVER, data can be created via the command line tool.
>> >
>> > import java.io.UnsupportedEncodingException;
>> > import java.util.List;
>> >
>> > import org.apache.cassandra.thrift.Cassandra;
>> > import org.apache.cassandra.thrift.Column;
>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>> > import org.apache.cassandra.thrift.ColumnParent;
>> > import org.apache.cassandra.thrift.ColumnPath;
>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>> > import org.apache.cassandra.thrift.InvalidRequestException;
>> > import org.apache.cassandra.thrift.NotFoundException;
>> > import org.apache.cassandra.thrift.SlicePredicate;
>> > import org.apache.cassandra.thrift.SliceRange;
>> > import org.apache.cassandra.thrift.SuperColumn;
>> > import org.apache.cassandra.thrift.TimedOutException;
>> > import org.apache.cassandra.thrift.UnavailableException;
>> > import org.apache.thrift.TException; import
>> > org.apache.thrift.protocol.TBinaryProtocol;
>> > import org.apache.thrift.protocol.TProtocol;
>> > import org.apache.thrift.transport.TSocket;
>> > import org.apache.thrift.transport.TTransport;
>> >
>> >
>> > public class CrudTest {
>> >
>> >
>> >        private static final String KEYSPACE = "Keyspace1";
>> >
>> >
>> >        public static void main(String[] args) {
>> >                CrudTest client = new CrudTest();
>> >
>> >                try {
>> >                        client.run();
>> >                } catch (Exception e) {
>> >                        e.printStackTrace();
>> >                }
>> >
>> >        }
>> >
>> >
>> >        public void run() throws TException,
>> > InvalidRequestException, UnavailableException,
>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>> > {
>> >                TTransport tr = new TSocket("localhost", 9160);
>> >                TProtocol proto = new TBinaryProtocol(tr);
>> >                Cassandra.Client client = new
>> > Cassandra.Client(proto);
>> >                tr.open();
>> >
>> >                System.out.println("******** CREATING DATA
>> > *********");
>> >                createData(client);
>> >                getData(client);
>> >                System.out.println();
>> >                System.out.println("******** DELETING DATA
>> > *********");
>> >                deleteData(client);
>> >                getData(client);
>> >                System.out.println();
>> >                System.out.println("******** CREATING DATA
>> > *********");
>> >                createData(client);
>> >                getData(client);
>> >
>> >                tr.close();
>> >          }
>> >
>> >
>> >        private void createData(Cassandra.Client client) throws
>> > InvalidRequestException, UnavailableException, TimedOutException,
>> > TException {
>> >                ColumnPath cp1 = new ColumnPath("Super2");
>> >                cp1.setSuper_column("hotel".getBytes());
>> >                cp1.setColumn("Best Western".getBytes());
>> >
>> >
>> >                client.insert(KEYSPACE,
>> >                          "name",
>> >                          cp1,
>> >                          "Best Western of SF".getBytes(),
>> >                          System.currentTimeMillis(),
>> >                          ConsistencyLevel.ALL);
>> >
>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >                cp2.setSuper_column("hotel".getBytes());
>> >                cp2.setColumn("Econolodge".getBytes());
>> >
>> >                client.insert(KEYSPACE,
>> >                                  "name",
>> >                                  cp2,
>> >                                  "Econolodge of SF".getBytes(),
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ALL);
>> >
>> >        }
>> >
>> >
>> >        private void deleteData(Cassandra.Client client) throws
>> > InvalidRequestException, UnavailableException, TimedOutException,
>> > TException {
>> >
>> >                client.remove(KEYSPACE,
>> >                                  "hotel",
>> >                                  new ColumnPath("Super2"),
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ONE);
>> >
>> >        }
>> >
>> >
>> >        private void getData(Cassandra.Client client) throws
>> > InvalidRequestException, UnavailableException, TimedOutException,
>> > TException {
>> >                SliceRange sliceRange = new SliceRange();
>> >                sliceRange.setStart(new byte[] {});
>> >                sliceRange.setFinish(new byte[] {});
>> >
>> >                SlicePredicate slicePredicate = new
>> > SlicePredicate();
>> >                slicePredicate.setSlice_range(sliceRange);
>> >
>> >                getData(client, slicePredicate);
>> >        }
>> >
>> >
>> >        private void getData(Cassandra.Client client, SlicePredicate
>> > slicePredicate) throws InvalidRequestException,
>> > UnavailableException, TimedOutException, TException {
>> >                List<ColumnOrSuperColumn> coscList =
>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>> > slicePredicate, ConsistencyLevel.ALL);
>> >
>> >                if (coscList.isEmpty()) {
>> >                        System.out.println("Column Or Super Column
>> > is EMPTY");
>> >                }
>> >
>> >                for (ColumnOrSuperColumn cosc: coscList) {
>> >
>> >                        if (cosc == null) {
>> >                                System.out.println("NULL RETURN
>> > VALUE");
>> >                        }
>> >
>> >                        SuperColumn superColumn =
>> > cosc.getSuper_column();
>> >
>> >                        if (superColumn == null) {
>> >                                System.out.println("Super Column is
>> > NULL");
>> >                        }
>> >                        else {
>> >                                showSuperColumnInfo(superColumn);
>> >                        }
>> >
>> >                }
>> >
>> >        }
>> >
>> >
>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>> >                System.out.println("######## Super Columns
>> > ###########");
>> >                System.out.println("Super Column Name = " + new
>> > String(superColumn.getName()));
>> >                List<Column> columnList = superColumn.getColumns();
>> >
>> >                System.out.println("--------- Start Columns
>> > -----------");
>> >                for (Column column: columnList) {
>> >                        System.out.println("Column Name = " + new
>> > String(column.getName()));
>> >                        System.out.println("Column Value = " + new
>> > String(column.getValue()));
>> >                }
>> >                System.out.println("--------- End Columns
>> > -----------");
>> >                
>> > System.out.println("##################################");
>> >        }
>> >
>> >
>> > }
>> >
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Oleg Tsvinev [mailto:[hidden email]]
>> > Sent: Thu 7/22/2010 1:56 PM
>> > To: [hidden email]
>> > Subject: Re: CRUD test
>> >
>> > Yes, and that was the issue. But now after I delete a row from
>> > cassandra-cli, I cannot insert anything back with my code. Insert
>> > code works does not throw any exceptions but when I read just
>> > inserted columns I get NotFoundException at the last line:
>> >
>> >            client = borrowClient();
>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>> > CONSISTENCY_LEVEL);
>> >            ColumnPath cp = new ColumnPath(application);
>> >            cp.setSuper_column(uuid.getBytes());
>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>> >
>> > It makes me think that once I remove supercolumn it cannot be
>> > created again.
>> >
>> >
>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:
>> >
>> > Have you checked the timestamp you're using for the subsequent
>> > inserts is higher than that used in the delete?
>> >
>> >
>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>> > <[hidden email]>
>> > wrote:
>> >> Hi there,
>> >> I'm try...
>> > --
>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>> > chandelier wearing a boar's head.
>> >
>> > Something which you, I, and everyone else would call "Tuesday", of
>> > course.
>> >
>> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Jonathan Shook
Correct.

After the initial insert,

cassandra> get Keyspace1.Super2['name']
=> (super_column=hotel,
     (column=Best Western, value=Best Western of SF, timestamp=1279916772571)
     (column=Econolodge, value=Econolodge of SF, timestamp=1279916772573))
Returned 1 results.

... and ...

cassandra> get Keyspace1.Super2['hotel']
Returned 0 results.



On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo
<[hidden email]> wrote:

> The Model Should look like:
>
>
> Super2 = {
>        hotel: {
>                        Best Western: {name: "Best Western of SF"}
>                        Econolodge: {name: "Econolodge of SF"}
>        }
> }
>
> Are the CRUD Operations not referencing this correctly?
>
>
>
> -----Original Message-----
> From: Jonathan Shook [mailto:[hidden email]]
> Sent: Friday, July 23, 2010 1:34 PM
> To: [hidden email]
> Subject: Re: CRUD test
>
> There seem to be data consistency bugs in the test.  Are "name" and "hotel" being used in a pair-wise way?
> Specifically, the first test is using creating one and checking for the other.
>
> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]> wrote:
>> Johathan,
>> I followed your suggestion. Unfortunately, CRUD test still does not
>> work for me. Can you provide a simplest CRUD test possible that works?
>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]> wrote:
>>>
>>> I suspect that it is still your timestamps.
>>> You can verify this with a fake timestamp generator that is simply
>>> incremented on each getTimestamp().
>>>
>>> 1 millisecond is a long time for code that is wrapped tightly in a
>>> test. You are likely using the same logical time stamp for multiple
>>> operations.
>>>
>>>
>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>>> <[hidden email]> wrote:
>>> > I am able to reproduce his problem. If you take the default
>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>>> > the code below.  You will see that the data is not getting created
>>> > once you run the delete.  It seems to not allow you to create data
>>> > via Thrift.  HOWEVER, data can be created via the command line tool.
>>> >
>>> > import java.io.UnsupportedEncodingException;
>>> > import java.util.List;
>>> >
>>> > import org.apache.cassandra.thrift.Cassandra;
>>> > import org.apache.cassandra.thrift.Column;
>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>>> > import org.apache.cassandra.thrift.ColumnParent;
>>> > import org.apache.cassandra.thrift.ColumnPath;
>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>>> > import org.apache.cassandra.thrift.NotFoundException;
>>> > import org.apache.cassandra.thrift.SlicePredicate;
>>> > import org.apache.cassandra.thrift.SliceRange;
>>> > import org.apache.cassandra.thrift.SuperColumn;
>>> > import org.apache.cassandra.thrift.TimedOutException;
>>> > import org.apache.cassandra.thrift.UnavailableException;
>>> > import org.apache.thrift.TException; import
>>> > org.apache.thrift.protocol.TBinaryProtocol;
>>> > import org.apache.thrift.protocol.TProtocol;
>>> > import org.apache.thrift.transport.TSocket;
>>> > import org.apache.thrift.transport.TTransport;
>>> >
>>> >
>>> > public class CrudTest {
>>> >
>>> >
>>> >        private static final String KEYSPACE = "Keyspace1";
>>> >
>>> >
>>> >        public static void main(String[] args) {
>>> >                CrudTest client = new CrudTest();
>>> >
>>> >                try {
>>> >                        client.run();
>>> >                } catch (Exception e) {
>>> >                        e.printStackTrace();
>>> >                }
>>> >
>>> >        }
>>> >
>>> >
>>> >        public void run() throws TException,
>>> > InvalidRequestException, UnavailableException,
>>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>>> > {
>>> >                TTransport tr = new TSocket("localhost", 9160);
>>> >                TProtocol proto = new TBinaryProtocol(tr);
>>> >                Cassandra.Client client = new
>>> > Cassandra.Client(proto);
>>> >                tr.open();
>>> >
>>> >                System.out.println("******** CREATING DATA
>>> > *********");
>>> >                createData(client);
>>> >                getData(client);
>>> >                System.out.println();
>>> >                System.out.println("******** DELETING DATA
>>> > *********");
>>> >                deleteData(client);
>>> >                getData(client);
>>> >                System.out.println();
>>> >                System.out.println("******** CREATING DATA
>>> > *********");
>>> >                createData(client);
>>> >                getData(client);
>>> >
>>> >                tr.close();
>>> >          }
>>> >
>>> >
>>> >        private void createData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>>> >                cp1.setSuper_column("hotel".getBytes());
>>> >                cp1.setColumn("Best Western".getBytes());
>>> >
>>> >
>>> >                client.insert(KEYSPACE,
>>> >                          "name",
>>> >                          cp1,
>>> >                          "Best Western of SF".getBytes(),
>>> >                          System.currentTimeMillis(),
>>> >                          ConsistencyLevel.ALL);
>>> >
>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>>> >                cp2.setSuper_column("hotel".getBytes());
>>> >                cp2.setColumn("Econolodge".getBytes());
>>> >
>>> >                client.insert(KEYSPACE,
>>> >                                  "name",
>>> >                                  cp2,
>>> >                                  "Econolodge of SF".getBytes(),
>>> >                                  System.currentTimeMillis(),
>>> >                                  ConsistencyLevel.ALL);
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void deleteData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >
>>> >                client.remove(KEYSPACE,
>>> >                                  "hotel",
>>> >                                  new ColumnPath("Super2"),
>>> >                                  System.currentTimeMillis(),
>>> >                                  ConsistencyLevel.ONE);
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void getData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >                SliceRange sliceRange = new SliceRange();
>>> >                sliceRange.setStart(new byte[] {});
>>> >                sliceRange.setFinish(new byte[] {});
>>> >
>>> >                SlicePredicate slicePredicate = new
>>> > SlicePredicate();
>>> >                slicePredicate.setSlice_range(sliceRange);
>>> >
>>> >                getData(client, slicePredicate);
>>> >        }
>>> >
>>> >
>>> >        private void getData(Cassandra.Client client, SlicePredicate
>>> > slicePredicate) throws InvalidRequestException,
>>> > UnavailableException, TimedOutException, TException {
>>> >                List<ColumnOrSuperColumn> coscList =
>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>>> > slicePredicate, ConsistencyLevel.ALL);
>>> >
>>> >                if (coscList.isEmpty()) {
>>> >                        System.out.println("Column Or Super Column
>>> > is EMPTY");
>>> >                }
>>> >
>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>>> >
>>> >                        if (cosc == null) {
>>> >                                System.out.println("NULL RETURN
>>> > VALUE");
>>> >                        }
>>> >
>>> >                        SuperColumn superColumn =
>>> > cosc.getSuper_column();
>>> >
>>> >                        if (superColumn == null) {
>>> >                                System.out.println("Super Column is
>>> > NULL");
>>> >                        }
>>> >                        else {
>>> >                                showSuperColumnInfo(superColumn);
>>> >                        }
>>> >
>>> >                }
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>>> >                System.out.println("######## Super Columns
>>> > ###########");
>>> >                System.out.println("Super Column Name = " + new
>>> > String(superColumn.getName()));
>>> >                List<Column> columnList = superColumn.getColumns();
>>> >
>>> >                System.out.println("--------- Start Columns
>>> > -----------");
>>> >                for (Column column: columnList) {
>>> >                        System.out.println("Column Name = " + new
>>> > String(column.getName()));
>>> >                        System.out.println("Column Value = " + new
>>> > String(column.getValue()));
>>> >                }
>>> >                System.out.println("--------- End Columns
>>> > -----------");
>>> >
>>> > System.out.println("##################################");
>>> >        }
>>> >
>>> >
>>> > }
>>> >
>>> >
>>> >
>>> >
>>> > -----Original Message-----
>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>>> > Sent: Thu 7/22/2010 1:56 PM
>>> > To: [hidden email]
>>> > Subject: Re: CRUD test
>>> >
>>> > Yes, and that was the issue. But now after I delete a row from
>>> > cassandra-cli, I cannot insert anything back with my code. Insert
>>> > code works does not throw any exceptions but when I read just
>>> > inserted columns I get NotFoundException at the last line:
>>> >
>>> >            client = borrowClient();
>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>>> > CONSISTENCY_LEVEL);
>>> >            ColumnPath cp = new ColumnPath(application);
>>> >            cp.setSuper_column(uuid.getBytes());
>>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>>> >
>>> > It makes me think that once I remove supercolumn it cannot be
>>> > created again.
>>> >
>>> >
>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:
>>> >
>>> > Have you checked the timestamp you're using for the subsequent
>>> > inserts is higher than that used in the delete?
>>> >
>>> >
>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>>> > <[hidden email]>
>>> > wrote:
>>> >> Hi there,
>>> >> I'm try...
>>> > --
>>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>>> > chandelier wearing a boar's head.
>>> >
>>> > Something which you, I, and everyone else would call "Tuesday", of
>>> > course.
>>> >
>>> >
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

RE: CRUD test

Peter Minearo
Interesting!! Let me rephrase to make sure I understood what is going on:

When Inserting data via the "insert" function/method:

void insert(string keyspace, string key, ColumnPath column_path, binary value, i64 timestamp, ConsistencyLevel consistency_level)

The "key" parameter is the actual Key to the "Row", which contains SuperColumns.  The 'ColumnPath' gives the path within the Key.



INCORRECT:
ColumnPath cp1 = new ColumnPath("Super2");
cp1.setSuper_column("hotel".getBytes());
cp1.setColumn("Best Western".getBytes());
client.insert(KEYSPACE, "name", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);


CORRECT:
ColumnPath cp1 = new ColumnPath("Super2");
cp1.setSuper_column("name".getBytes());
cp1.setColumn("Best Western".getBytes());
client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);



 

-----Original Message-----
From: Jonathan Shook [mailto:[hidden email]]
Sent: Friday, July 23, 2010 1:49 PM
To: [hidden email]
Subject: Re: CRUD test

Correct.

After the initial insert,

cassandra> get Keyspace1.Super2['name']
=> (super_column=hotel,
     (column=Best Western, value=Best Western of SF, timestamp=1279916772571)
     (column=Econolodge, value=Econolodge of SF, timestamp=1279916772573)) Returned 1 results.

... and ...

cassandra> get Keyspace1.Super2['hotel']
Returned 0 results.



On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo <[hidden email]> wrote:

> The Model Should look like:
>
>
> Super2 = {
>        hotel: {
>                        Best Western: {name: "Best Western of SF"}
>                        Econolodge: {name: "Econolodge of SF"}
>        }
> }
>
> Are the CRUD Operations not referencing this correctly?
>
>
>
> -----Original Message-----
> From: Jonathan Shook [mailto:[hidden email]]
> Sent: Friday, July 23, 2010 1:34 PM
> To: [hidden email]
> Subject: Re: CRUD test
>
> There seem to be data consistency bugs in the test.  Are "name" and "hotel" being used in a pair-wise way?
> Specifically, the first test is using creating one and checking for the other.
>
> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]> wrote:
>> Johathan,
>> I followed your suggestion. Unfortunately, CRUD test still does not
>> work for me. Can you provide a simplest CRUD test possible that works?
>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]> wrote:
>>>
>>> I suspect that it is still your timestamps.
>>> You can verify this with a fake timestamp generator that is simply
>>> incremented on each getTimestamp().
>>>
>>> 1 millisecond is a long time for code that is wrapped tightly in a
>>> test. You are likely using the same logical time stamp for multiple
>>> operations.
>>>
>>>
>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>>> <[hidden email]> wrote:
>>> > I am able to reproduce his problem. If you take the default
>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>>> > the code below.  You will see that the data is not getting created
>>> > once you run the delete.  It seems to not allow you to create data
>>> > via Thrift.  HOWEVER, data can be created via the command line tool.
>>> >
>>> > import java.io.UnsupportedEncodingException;
>>> > import java.util.List;
>>> >
>>> > import org.apache.cassandra.thrift.Cassandra;
>>> > import org.apache.cassandra.thrift.Column;
>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>>> > import org.apache.cassandra.thrift.ColumnParent;
>>> > import org.apache.cassandra.thrift.ColumnPath;
>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>>> > import org.apache.cassandra.thrift.NotFoundException;
>>> > import org.apache.cassandra.thrift.SlicePredicate;
>>> > import org.apache.cassandra.thrift.SliceRange;
>>> > import org.apache.cassandra.thrift.SuperColumn;
>>> > import org.apache.cassandra.thrift.TimedOutException;
>>> > import org.apache.cassandra.thrift.UnavailableException;
>>> > import org.apache.thrift.TException; import
>>> > org.apache.thrift.protocol.TBinaryProtocol;
>>> > import org.apache.thrift.protocol.TProtocol;
>>> > import org.apache.thrift.transport.TSocket;
>>> > import org.apache.thrift.transport.TTransport;
>>> >
>>> >
>>> > public class CrudTest {
>>> >
>>> >
>>> >        private static final String KEYSPACE = "Keyspace1";
>>> >
>>> >
>>> >        public static void main(String[] args) {
>>> >                CrudTest client = new CrudTest();
>>> >
>>> >                try {
>>> >                        client.run();
>>> >                } catch (Exception e) {
>>> >                        e.printStackTrace();
>>> >                }
>>> >
>>> >        }
>>> >
>>> >
>>> >        public void run() throws TException,
>>> > InvalidRequestException, UnavailableException,
>>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>>> > {
>>> >                TTransport tr = new TSocket("localhost", 9160);
>>> >                TProtocol proto = new TBinaryProtocol(tr);
>>> >                Cassandra.Client client = new
>>> > Cassandra.Client(proto);
>>> >                tr.open();
>>> >
>>> >                System.out.println("******** CREATING DATA
>>> > *********");
>>> >                createData(client);
>>> >                getData(client);
>>> >                System.out.println();
>>> >                System.out.println("******** DELETING DATA
>>> > *********");
>>> >                deleteData(client);
>>> >                getData(client);
>>> >                System.out.println();
>>> >                System.out.println("******** CREATING DATA
>>> > *********");
>>> >                createData(client);
>>> >                getData(client);
>>> >
>>> >                tr.close();
>>> >          }
>>> >
>>> >
>>> >        private void createData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>>> >                cp1.setSuper_column("hotel".getBytes());
>>> >                cp1.setColumn("Best Western".getBytes());
>>> >
>>> >
>>> >                client.insert(KEYSPACE,
>>> >                          "name",
>>> >                          cp1,
>>> >                          "Best Western of SF".getBytes(),
>>> >                          System.currentTimeMillis(),
>>> >                          ConsistencyLevel.ALL);
>>> >
>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>>> >                cp2.setSuper_column("hotel".getBytes());
>>> >                cp2.setColumn("Econolodge".getBytes());
>>> >
>>> >                client.insert(KEYSPACE,
>>> >                                  "name",
>>> >                                  cp2,
>>> >                                  "Econolodge of SF".getBytes(),
>>> >                                  System.currentTimeMillis(),
>>> >                                  ConsistencyLevel.ALL);
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void deleteData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >
>>> >                client.remove(KEYSPACE,
>>> >                                  "hotel",
>>> >                                  new ColumnPath("Super2"),
>>> >                                  System.currentTimeMillis(),
>>> >                                  ConsistencyLevel.ONE);
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void getData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >                SliceRange sliceRange = new SliceRange();
>>> >                sliceRange.setStart(new byte[] {});
>>> >                sliceRange.setFinish(new byte[] {});
>>> >
>>> >                SlicePredicate slicePredicate = new
>>> > SlicePredicate();
>>> >                slicePredicate.setSlice_range(sliceRange);
>>> >
>>> >                getData(client, slicePredicate);
>>> >        }
>>> >
>>> >
>>> >        private void getData(Cassandra.Client client,
>>> > SlicePredicate
>>> > slicePredicate) throws InvalidRequestException,
>>> > UnavailableException, TimedOutException, TException {
>>> >                List<ColumnOrSuperColumn> coscList =
>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>>> > slicePredicate, ConsistencyLevel.ALL);
>>> >
>>> >                if (coscList.isEmpty()) {
>>> >                        System.out.println("Column Or Super Column
>>> > is EMPTY");
>>> >                }
>>> >
>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>>> >
>>> >                        if (cosc == null) {
>>> >                                System.out.println("NULL RETURN
>>> > VALUE");
>>> >                        }
>>> >
>>> >                        SuperColumn superColumn =
>>> > cosc.getSuper_column();
>>> >
>>> >                        if (superColumn == null) {
>>> >                                System.out.println("Super Column is
>>> > NULL");
>>> >                        }
>>> >                        else {
>>> >                                showSuperColumnInfo(superColumn);
>>> >                        }
>>> >
>>> >                }
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>>> >                System.out.println("######## Super Columns
>>> > ###########");
>>> >                System.out.println("Super Column Name = " + new
>>> > String(superColumn.getName()));
>>> >                List<Column> columnList = superColumn.getColumns();
>>> >
>>> >                System.out.println("--------- Start Columns
>>> > -----------");
>>> >                for (Column column: columnList) {
>>> >                        System.out.println("Column Name = " + new
>>> > String(column.getName()));
>>> >                        System.out.println("Column Value = " + new
>>> > String(column.getValue()));
>>> >                }
>>> >                System.out.println("--------- End Columns
>>> > -----------");
>>> >
>>> > System.out.println("##################################");
>>> >        }
>>> >
>>> >
>>> > }
>>> >
>>> >
>>> >
>>> >
>>> > -----Original Message-----
>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>>> > Sent: Thu 7/22/2010 1:56 PM
>>> > To: [hidden email]
>>> > Subject: Re: CRUD test
>>> >
>>> > Yes, and that was the issue. But now after I delete a row from
>>> > cassandra-cli, I cannot insert anything back with my code. Insert
>>> > code works does not throw any exceptions but when I read just
>>> > inserted columns I get NotFoundException at the last line:
>>> >
>>> >            client = borrowClient();
>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>>> > CONSISTENCY_LEVEL);
>>> >            ColumnPath cp = new ColumnPath(application);
>>> >            cp.setSuper_column(uuid.getBytes());
>>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>>> >
>>> > It makes me think that once I remove supercolumn it cannot be
>>> > created again.
>>> >
>>> >
>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:
>>> >
>>> > Have you checked the timestamp you're using for the subsequent
>>> > inserts is higher than that used in the delete?
>>> >
>>> >
>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>>> > <[hidden email]>
>>> > wrote:
>>> >> Hi there,
>>> >> I'm try...
>>> > --
>>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>>> > chandelier wearing a boar's head.
>>> >
>>> > Something which you, I, and everyone else would call "Tuesday", of
>>> > course.
>>> >
>>> >
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

RE: CRUD test

Peter Minearo
CORRECTION:

ColumnPath cp1 = new ColumnPath("Super2");
cp1.setSuper_column("Best Western".getBytes());
cp1.setColumn("name".getBytes());
client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);
 

-----Original Message-----
From: Peter Minearo [mailto:[hidden email]]
Sent: Friday, July 23, 2010 2:14 PM
To: [hidden email]
Subject: RE: CRUD test

Interesting!! Let me rephrase to make sure I understood what is going on:

When Inserting data via the "insert" function/method:

void insert(string keyspace, string key, ColumnPath column_path, binary value, i64 timestamp, ConsistencyLevel consistency_level)

The "key" parameter is the actual Key to the "Row", which contains SuperColumns.  The 'ColumnPath' gives the path within the Key.



INCORRECT:
ColumnPath cp1 = new ColumnPath("Super2"); cp1.setSuper_column("hotel".getBytes());
cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE, "name", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);


CORRECT:
ColumnPath cp1 = new ColumnPath("Super2"); cp1.setSuper_column("name".getBytes());
cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);



 

-----Original Message-----
From: Jonathan Shook [mailto:[hidden email]]
Sent: Friday, July 23, 2010 1:49 PM
To: [hidden email]
Subject: Re: CRUD test

Correct.

After the initial insert,

cassandra> get Keyspace1.Super2['name']
=> (super_column=hotel,
     (column=Best Western, value=Best Western of SF, timestamp=1279916772571)
     (column=Econolodge, value=Econolodge of SF, timestamp=1279916772573)) Returned 1 results.

... and ...

cassandra> get Keyspace1.Super2['hotel']
Returned 0 results.



On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo <[hidden email]> wrote:

> The Model Should look like:
>
>
> Super2 = {
>        hotel: {
>                        Best Western: {name: "Best Western of SF"}
>                        Econolodge: {name: "Econolodge of SF"}
>        }
> }
>
> Are the CRUD Operations not referencing this correctly?
>
>
>
> -----Original Message-----
> From: Jonathan Shook [mailto:[hidden email]]
> Sent: Friday, July 23, 2010 1:34 PM
> To: [hidden email]
> Subject: Re: CRUD test
>
> There seem to be data consistency bugs in the test.  Are "name" and "hotel" being used in a pair-wise way?
> Specifically, the first test is using creating one and checking for the other.
>
> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]> wrote:
>> Johathan,
>> I followed your suggestion. Unfortunately, CRUD test still does not
>> work for me. Can you provide a simplest CRUD test possible that works?
>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]> wrote:
>>>
>>> I suspect that it is still your timestamps.
>>> You can verify this with a fake timestamp generator that is simply
>>> incremented on each getTimestamp().
>>>
>>> 1 millisecond is a long time for code that is wrapped tightly in a
>>> test. You are likely using the same logical time stamp for multiple
>>> operations.
>>>
>>>
>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>>> <[hidden email]> wrote:
>>> > I am able to reproduce his problem. If you take the default
>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>>> > the code below.  You will see that the data is not getting created
>>> > once you run the delete.  It seems to not allow you to create data
>>> > via Thrift.  HOWEVER, data can be created via the command line tool.
>>> >
>>> > import java.io.UnsupportedEncodingException;
>>> > import java.util.List;
>>> >
>>> > import org.apache.cassandra.thrift.Cassandra;
>>> > import org.apache.cassandra.thrift.Column;
>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>>> > import org.apache.cassandra.thrift.ColumnParent;
>>> > import org.apache.cassandra.thrift.ColumnPath;
>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>>> > import org.apache.cassandra.thrift.NotFoundException;
>>> > import org.apache.cassandra.thrift.SlicePredicate;
>>> > import org.apache.cassandra.thrift.SliceRange;
>>> > import org.apache.cassandra.thrift.SuperColumn;
>>> > import org.apache.cassandra.thrift.TimedOutException;
>>> > import org.apache.cassandra.thrift.UnavailableException;
>>> > import org.apache.thrift.TException; import
>>> > org.apache.thrift.protocol.TBinaryProtocol;
>>> > import org.apache.thrift.protocol.TProtocol;
>>> > import org.apache.thrift.transport.TSocket;
>>> > import org.apache.thrift.transport.TTransport;
>>> >
>>> >
>>> > public class CrudTest {
>>> >
>>> >
>>> >        private static final String KEYSPACE = "Keyspace1";
>>> >
>>> >
>>> >        public static void main(String[] args) {
>>> >                CrudTest client = new CrudTest();
>>> >
>>> >                try {
>>> >                        client.run();
>>> >                } catch (Exception e) {
>>> >                        e.printStackTrace();
>>> >                }
>>> >
>>> >        }
>>> >
>>> >
>>> >        public void run() throws TException,
>>> > InvalidRequestException, UnavailableException,
>>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>>> > {
>>> >                TTransport tr = new TSocket("localhost", 9160);
>>> >                TProtocol proto = new TBinaryProtocol(tr);
>>> >                Cassandra.Client client = new
>>> > Cassandra.Client(proto);
>>> >                tr.open();
>>> >
>>> >                System.out.println("******** CREATING DATA
>>> > *********");
>>> >                createData(client);
>>> >                getData(client);
>>> >                System.out.println();
>>> >                System.out.println("******** DELETING DATA
>>> > *********");
>>> >                deleteData(client);
>>> >                getData(client);
>>> >                System.out.println();
>>> >                System.out.println("******** CREATING DATA
>>> > *********");
>>> >                createData(client);
>>> >                getData(client);
>>> >
>>> >                tr.close();
>>> >          }
>>> >
>>> >
>>> >        private void createData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>>> >                cp1.setSuper_column("hotel".getBytes());
>>> >                cp1.setColumn("Best Western".getBytes());
>>> >
>>> >
>>> >                client.insert(KEYSPACE,
>>> >                          "name",
>>> >                          cp1,
>>> >                          "Best Western of SF".getBytes(),
>>> >                          System.currentTimeMillis(),
>>> >                          ConsistencyLevel.ALL);
>>> >
>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>>> >                cp2.setSuper_column("hotel".getBytes());
>>> >                cp2.setColumn("Econolodge".getBytes());
>>> >
>>> >                client.insert(KEYSPACE,
>>> >                                  "name",
>>> >                                  cp2,
>>> >                                  "Econolodge of SF".getBytes(),
>>> >                                  System.currentTimeMillis(),
>>> >                                  ConsistencyLevel.ALL);
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void deleteData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >
>>> >                client.remove(KEYSPACE,
>>> >                                  "hotel",
>>> >                                  new ColumnPath("Super2"),
>>> >                                  System.currentTimeMillis(),
>>> >                                  ConsistencyLevel.ONE);
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void getData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >                SliceRange sliceRange = new SliceRange();
>>> >                sliceRange.setStart(new byte[] {});
>>> >                sliceRange.setFinish(new byte[] {});
>>> >
>>> >                SlicePredicate slicePredicate = new
>>> > SlicePredicate();
>>> >                slicePredicate.setSlice_range(sliceRange);
>>> >
>>> >                getData(client, slicePredicate);
>>> >        }
>>> >
>>> >
>>> >        private void getData(Cassandra.Client client,
>>> > SlicePredicate
>>> > slicePredicate) throws InvalidRequestException,
>>> > UnavailableException, TimedOutException, TException {
>>> >                List<ColumnOrSuperColumn> coscList =
>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>>> > slicePredicate, ConsistencyLevel.ALL);
>>> >
>>> >                if (coscList.isEmpty()) {
>>> >                        System.out.println("Column Or Super Column
>>> > is EMPTY");
>>> >                }
>>> >
>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>>> >
>>> >                        if (cosc == null) {
>>> >                                System.out.println("NULL RETURN
>>> > VALUE");
>>> >                        }
>>> >
>>> >                        SuperColumn superColumn =
>>> > cosc.getSuper_column();
>>> >
>>> >                        if (superColumn == null) {
>>> >                                System.out.println("Super Column is
>>> > NULL");
>>> >                        }
>>> >                        else {
>>> >                                showSuperColumnInfo(superColumn);
>>> >                        }
>>> >
>>> >                }
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>>> >                System.out.println("######## Super Columns
>>> > ###########");
>>> >                System.out.println("Super Column Name = " + new
>>> > String(superColumn.getName()));
>>> >                List<Column> columnList = superColumn.getColumns();
>>> >
>>> >                System.out.println("--------- Start Columns
>>> > -----------");
>>> >                for (Column column: columnList) {
>>> >                        System.out.println("Column Name = " + new
>>> > String(column.getName()));
>>> >                        System.out.println("Column Value = " + new
>>> > String(column.getValue()));
>>> >                }
>>> >                System.out.println("--------- End Columns
>>> > -----------");
>>> >
>>> > System.out.println("##################################");
>>> >        }
>>> >
>>> >
>>> > }
>>> >
>>> >
>>> >
>>> >
>>> > -----Original Message-----
>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>>> > Sent: Thu 7/22/2010 1:56 PM
>>> > To: [hidden email]
>>> > Subject: Re: CRUD test
>>> >
>>> > Yes, and that was the issue. But now after I delete a row from
>>> > cassandra-cli, I cannot insert anything back with my code. Insert
>>> > code works does not throw any exceptions but when I read just
>>> > inserted columns I get NotFoundException at the last line:
>>> >
>>> >            client = borrowClient();
>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>>> > CONSISTENCY_LEVEL);
>>> >            ColumnPath cp = new ColumnPath(application);
>>> >            cp.setSuper_column(uuid.getBytes());
>>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>>> >
>>> > It makes me think that once I remove supercolumn it cannot be
>>> > created again.
>>> >
>>> >
>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:
>>> >
>>> > Have you checked the timestamp you're using for the subsequent
>>> > inserts is higher than that used in the delete?
>>> >
>>> >
>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>>> > <[hidden email]>
>>> > wrote:
>>> >> Hi there,
>>> >> I'm try...
>>> > --
>>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>>> > chandelier wearing a boar's head.
>>> >
>>> > Something which you, I, and everyone else would call "Tuesday", of
>>> > course.
>>> >
>>> >
>>
>>
>
Reply | Threaded
Open this post in threaded view
|

RE: CRUD test

Peter Minearo
Consequentially the remove should look like:


ColumnPath cp1 = new ColumnPath("Super2");
                cp1.setSuper_column("Best Western".getBytes());
               
                client.remove(KEYSPACE,
                                  "hotel",
                                  cp1,
                                  System.currentTimeMillis(),
                                  ConsistencyLevel.ONE);
               
                ColumnPath cp2 = new ColumnPath("Super2");
                cp2.setSuper_column("Econolodge".getBytes());
               
                client.remove(KEYSPACE,
                                  "hotel",
                                  cp2,
                                  System.currentTimeMillis(),
                                  ConsistencyLevel.ONE);


-----Original Message-----
From: Peter Minearo [mailto:[hidden email]]
Sent: Fri 7/23/2010 2:17 PM
To: [hidden email]
Subject: RE: CRUD test
 
CORRECTION:

ColumnPath cp1 = new ColumnPath("Super2");
cp1.setSuper_column("Best Western".getBytes());
cp1.setColumn("name".getBytes());
client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);
 

-----Original Message-----
From: Peter Minearo [mailto:[hidden email]]
Sent: Friday, July 23, 2010 2:14 PM
To: [hidden email]
Subject: RE: CRUD test

Interesting!! Let me rephrase to make sure I understood what is going on:

When Inserting data via the "insert" function/method:

void insert(string keyspace, string key, ColumnPath column_path, binary value, i64 timestamp, ConsistencyLevel consistency_level)

The "key" parameter is the actual Key to the "Row", which contains SuperColumns.  The 'ColumnPath' gives the path within the Key.



INCORRECT:
ColumnPath cp1 = new ColumnPath("Super2"); cp1.setSuper_column("hotel".getBytes());
cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE, "name", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);


CORRECT:
ColumnPath cp1 = new ColumnPath("Super2"); cp1.setSuper_column("name".getBytes());
cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);



 

-----Original Message-----
From: Jonathan Shook [mailto:[hidden email]]
Sent: Friday, July 23, 2010 1:49 PM
To: [hidden email]
Subject: Re: CRUD test

Correct.

After the initial insert,

cassandra> get Keyspace1.Super2['name']
=> (super_column=hotel,
     (column=Best Western, value=Best Western of SF, timestamp=1279916772571)
     (column=Econolodge, value=Econolodge of SF, timestamp=1279916772573)) Returned 1 results.

... and ...

cassandra> get Keyspace1.Super2['hotel']
Returned 0 results.



On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo <[hidden email]> wrote:

> The Model Should look like:
>
>
> Super2 = {
>        hotel: {
>                        Best Western: {name: "Best Western of SF"}
>                        Econolodge: {name: "Econolodge of SF"}
>        }
> }
>
> Are the CRUD Operations not referencing this correctly?
>
>
>
> -----Original Message-----
> From: Jonathan Shook [mailto:[hidden email]]
> Sent: Friday, July 23, 2010 1:34 PM
> To: [hidden email]
> Subject: Re: CRUD test
>
> There seem to be data consistency bugs in the test.  Are "name" and "hotel" being used in a pair-wise way?
> Specifically, the first test is using creating one and checking for the other.
>
> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]> wrote:
>> Johathan,
>> I followed your suggestion. Unfortunately, CRUD test still does not
>> work for me. Can you provide a simplest CRUD test possible that works?
>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]> wrote:
>>>
>>> I suspect that it is still your timestamps.
>>> You can verify this with a fake timestamp generator that is simply
>>> incremented on each getTimestamp().
>>>
>>> 1 millisecond is a long time for code that is wrapped tightly in a
>>> test. You are likely using the same logical time stamp for multiple
>>> operations.
>>>
>>>
>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>>> <[hidden email]> wrote:
>>> > I am able to reproduce his problem. If you take the default
>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>>> > the code below.  You will see that the data is not getting created
>>> > once you run the delete.  It seems to not allow you to create data
>>> > via Thrift.  HOWEVER, data can be created via the command line tool.
>>> >
>>> > import java.io.UnsupportedEncodingException;
>>> > import java.util.List;
>>> >
>>> > import org.apache.cassandra.thrift.Cassandra;
>>> > import org.apache.cassandra.thrift.Column;
>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>>> > import org.apache.cassandra.thrift.ColumnParent;
>>> > import org.apache.cassandra.thrift.ColumnPath;
>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>>> > import org.apache.cassandra.thrift.NotFoundException;
>>> > import org.apache.cassandra.thrift.SlicePredicate;
>>> > import org.apache.cassandra.thrift.SliceRange;
>>> > import org.apache.cassandra.thrift.SuperColumn;
>>> > import org.apache.cassandra.thrift.TimedOutException;
>>> > import org.apache.cassandra.thrift.UnavailableException;
>>> > import org.apache.thrift.TException; import
>>> > org.apache.thrift.protocol.TBinaryProtocol;
>>> > import org.apache.thrift.protocol.TProtocol;
>>> > import org.apache.thrift.transport.TSocket;
>>> > import org.apache.thrift.transport.TTransport;
>>> >
>>> >
>>> > public class CrudTest {
>>> >
>>> >
>>> >        private static final String KEYSPACE = "Keyspace1";
>>> >
>>> >
>>> >        public static void main(String[] args) {
>>> >                CrudTest client = new CrudTest();
>>> >
>>> >                try {
>>> >                        client.run();
>>> >                } catch (Exception e) {
>>> >                        e.printStackTrace();
>>> >                }
>>> >
>>> >        }
>>> >
>>> >
>>> >        public void run() throws TException,
>>> > InvalidRequestException, UnavailableException,
>>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>>> > {
>>> >                TTransport tr = new TSocket("localhost", 9160);
>>> >                TProtocol proto = new TBinaryProtocol(tr);
>>> >                Cassandra.Client client = new
>>> > Cassandra.Client(proto);
>>> >                tr.open();
>>> >
>>> >                System.out.println("******** CREATING DATA
>>> > *********");
>>> >                createData(client);
>>> >                getData(client);
>>> >                System.out.println();
>>> >                System.out.println("******** DELETING DATA
>>> > *********");
>>> >                deleteData(client);
>>> >                getData(client);
>>> >                System.out.println();
>>> >                System.out.println("******** CREATING DATA
>>> > *********");
>>> >                createData(client);
>>> >                getData(client);
>>> >
>>> >                tr.close();
>>> >          }
>>> >
>>> >
>>> >        private void createData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>>> >                cp1.setSuper_column("hotel".getBytes());
>>> >                cp1.setColumn("Best Western".getBytes());
>>> >
>>> >
>>> >                client.insert(KEYSPACE,
>>> >                          "name",
>>> >                          cp1,
>>> >                          "Best Western of SF".getBytes(),
>>> >                          System.currentTimeMillis(),
>>> >                          ConsistencyLevel.ALL);
>>> >
>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>>> >                cp2.setSuper_column("hotel".getBytes());
>>> >                cp2.setColumn("Econolodge".getBytes());
>>> >
>>> >                client.insert(KEYSPACE,
>>> >                                  "name",
>>> >                                  cp2,
>>> >                                  "Econolodge of SF".getBytes(),
>>> >                                  System.currentTimeMillis(),
>>> >                                  ConsistencyLevel.ALL);
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void deleteData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >
>>> >                client.remove(KEYSPACE,
>>> >                                  "hotel",
>>> >                                  new ColumnPath("Super2"),
>>> >                                  System.currentTimeMillis(),
>>> >                                  ConsistencyLevel.ONE);
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void getData(Cassandra.Client client) throws
>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>> > TException {
>>> >                SliceRange sliceRange = new SliceRange();
>>> >                sliceRange.setStart(new byte[] {});
>>> >                sliceRange.setFinish(new byte[] {});
>>> >
>>> >                SlicePredicate slicePredicate = new
>>> > SlicePredicate();
>>> >                slicePredicate.setSlice_range(sliceRange);
>>> >
>>> >                getData(client, slicePredicate);
>>> >        }
>>> >
>>> >
>>> >        private void getData(Cassandra.Client client,
>>> > SlicePredicate
>>> > slicePredicate) throws InvalidRequestException,
>>> > UnavailableException, TimedOutException, TException {
>>> >                List<ColumnOrSuperColumn> coscList =
>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>>> > slicePredicate, ConsistencyLevel.ALL);
>>> >
>>> >                if (coscList.isEmpty()) {
>>> >                        System.out.println("Column Or Super Column
>>> > is EMPTY");
>>> >                }
>>> >
>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>>> >
>>> >                        if (cosc == null) {
>>> >                                System.out.println("NULL RETURN
>>> > VALUE");
>>> >                        }
>>> >
>>> >                        SuperColumn superColumn =
>>> > cosc.getSuper_column();
>>> >
>>> >                        if (superColumn == null) {
>>> >                                System.out.println("Super Column is
>>> > NULL");
>>> >                        }
>>> >                        else {
>>> >                                showSuperColumnInfo(superColumn);
>>> >                        }
>>> >
>>> >                }
>>> >
>>> >        }
>>> >
>>> >
>>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>>> >                System.out.println("######## Super Columns
>>> > ###########");
>>> >                System.out.println("Super Column Name = " + new
>>> > String(superColumn.getName()));
>>> >                List<Column> columnList = superColumn.getColumns();
>>> >
>>> >                System.out.println("--------- Start Columns
>>> > -----------");
>>> >                for (Column column: columnList) {
>>> >                        System.out.println("Column Name = " + new
>>> > String(column.getName()));
>>> >                        System.out.println("Column Value = " + new
>>> > String(column.getValue()));
>>> >                }
>>> >                System.out.println("--------- End Columns
>>> > -----------");
>>> >
>>> > System.out.println("##################################");
>>> >        }
>>> >
>>> >
>>> > }
>>> >
>>> >
>>> >
>>> >
>>> > -----Original Message-----
>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>>> > Sent: Thu 7/22/2010 1:56 PM
>>> > To: [hidden email]
>>> > Subject: Re: CRUD test
>>> >
>>> > Yes, and that was the issue. But now after I delete a row from
>>> > cassandra-cli, I cannot insert anything back with my code. Insert
>>> > code works does not throw any exceptions but when I read just
>>> > inserted columns I get NotFoundException at the last line:
>>> >
>>> >            client = borrowClient();
>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>>> > CONSISTENCY_LEVEL);
>>> >            ColumnPath cp = new ColumnPath(application);
>>> >            cp.setSuper_column(uuid.getBytes());
>>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>>> >
>>> > It makes me think that once I remove supercolumn it cannot be
>>> > created again.
>>> >
>>> >
>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:
>>> >
>>> > Have you checked the timestamp you're using for the subsequent
>>> > inserts is higher than that used in the delete?
>>> >
>>> >
>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>>> > <[hidden email]>
>>> > wrote:
>>> >> Hi there,
>>> >> I'm try...
>>> > --
>>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>>> > chandelier wearing a boar's head.
>>> >
>>> > Something which you, I, and everyone else would call "Tuesday", of
>>> > course.
>>> >
>>> >
>>
>>
>


winmail.dat (10K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Jonathan Shook
I think you are getting it.

As far as what means what at which level, it's really about using them
consistently in every case. The [row] key (or [row] key range) is a
top-level argument for all of the operations, since it is the "key" to
mapping the set of responsible nodes. The key is the part of the name
of any column which most affects how the load is apportioned in the
cluster, so it is used very early in request processing.


On Fri, Jul 23, 2010 at 4:22 PM, Peter Minearo
<[hidden email]> wrote:

> Consequentially the remove should look like:
>
>
> ColumnPath cp1 = new ColumnPath("Super2");
>                cp1.setSuper_column("Best Western".getBytes());
>
>                client.remove(KEYSPACE,
>                                  "hotel",
>                                  cp1,
>                                  System.currentTimeMillis(),
>                                  ConsistencyLevel.ONE);
>
>                ColumnPath cp2 = new ColumnPath("Super2");
>                cp2.setSuper_column("Econolodge".getBytes());
>
>                client.remove(KEYSPACE,
>                                  "hotel",
>                                  cp2,
>                                  System.currentTimeMillis(),
>                                  ConsistencyLevel.ONE);
>
>
> -----Original Message-----
> From: Peter Minearo [mailto:[hidden email]]
> Sent: Fri 7/23/2010 2:17 PM
> To: [hidden email]
> Subject: RE: CRUD test
>
> CORRECTION:
>
> ColumnPath cp1 = new ColumnPath("Super2");
> cp1.setSuper_column("Best Western".getBytes());
> cp1.setColumn("name".getBytes());
> client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);
>
>
> -----Original Message-----
> From: Peter Minearo [mailto:[hidden email]]
> Sent: Friday, July 23, 2010 2:14 PM
> To: [hidden email]
> Subject: RE: CRUD test
>
> Interesting!! Let me rephrase to make sure I understood what is going on:
>
> When Inserting data via the "insert" function/method:
>
> void insert(string keyspace, string key, ColumnPath column_path, binary value, i64 timestamp, ConsistencyLevel consistency_level)
>
> The "key" parameter is the actual Key to the "Row", which contains SuperColumns.  The 'ColumnPath' gives the path within the Key.
>
>
>
> INCORRECT:
> ColumnPath cp1 = new ColumnPath("Super2"); cp1.setSuper_column("hotel".getBytes());
> cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE, "name", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);
>
>
> CORRECT:
> ColumnPath cp1 = new ColumnPath("Super2"); cp1.setSuper_column("name".getBytes());
> cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);
>
>
>
>
>
> -----Original Message-----
> From: Jonathan Shook [mailto:[hidden email]]
> Sent: Friday, July 23, 2010 1:49 PM
> To: [hidden email]
> Subject: Re: CRUD test
>
> Correct.
>
> After the initial insert,
>
> cassandra> get Keyspace1.Super2['name']
> => (super_column=hotel,
>     (column=Best Western, value=Best Western of SF, timestamp=1279916772571)
>     (column=Econolodge, value=Econolodge of SF, timestamp=1279916772573)) Returned 1 results.
>
> ... and ...
>
> cassandra> get Keyspace1.Super2['hotel']
> Returned 0 results.
>
>
>
> On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo <[hidden email]> wrote:
>> The Model Should look like:
>>
>>
>> Super2 = {
>>        hotel: {
>>                        Best Western: {name: "Best Western of SF"}
>>                        Econolodge: {name: "Econolodge of SF"}
>>        }
>> }
>>
>> Are the CRUD Operations not referencing this correctly?
>>
>>
>>
>> -----Original Message-----
>> From: Jonathan Shook [mailto:[hidden email]]
>> Sent: Friday, July 23, 2010 1:34 PM
>> To: [hidden email]
>> Subject: Re: CRUD test
>>
>> There seem to be data consistency bugs in the test.  Are "name" and "hotel" being used in a pair-wise way?
>> Specifically, the first test is using creating one and checking for the other.
>>
>> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]> wrote:
>>> Johathan,
>>> I followed your suggestion. Unfortunately, CRUD test still does not
>>> work for me. Can you provide a simplest CRUD test possible that works?
>>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]> wrote:
>>>>
>>>> I suspect that it is still your timestamps.
>>>> You can verify this with a fake timestamp generator that is simply
>>>> incremented on each getTimestamp().
>>>>
>>>> 1 millisecond is a long time for code that is wrapped tightly in a
>>>> test. You are likely using the same logical time stamp for multiple
>>>> operations.
>>>>
>>>>
>>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>>>> <[hidden email]> wrote:
>>>> > I am able to reproduce his problem. If you take the default
>>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>>>> > the code below.  You will see that the data is not getting created
>>>> > once you run the delete.  It seems to not allow you to create data
>>>> > via Thrift.  HOWEVER, data can be created via the command line tool.
>>>> >
>>>> > import java.io.UnsupportedEncodingException;
>>>> > import java.util.List;
>>>> >
>>>> > import org.apache.cassandra.thrift.Cassandra;
>>>> > import org.apache.cassandra.thrift.Column;
>>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>>>> > import org.apache.cassandra.thrift.ColumnParent;
>>>> > import org.apache.cassandra.thrift.ColumnPath;
>>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>>>> > import org.apache.cassandra.thrift.NotFoundException;
>>>> > import org.apache.cassandra.thrift.SlicePredicate;
>>>> > import org.apache.cassandra.thrift.SliceRange;
>>>> > import org.apache.cassandra.thrift.SuperColumn;
>>>> > import org.apache.cassandra.thrift.TimedOutException;
>>>> > import org.apache.cassandra.thrift.UnavailableException;
>>>> > import org.apache.thrift.TException; import
>>>> > org.apache.thrift.protocol.TBinaryProtocol;
>>>> > import org.apache.thrift.protocol.TProtocol;
>>>> > import org.apache.thrift.transport.TSocket;
>>>> > import org.apache.thrift.transport.TTransport;
>>>> >
>>>> >
>>>> > public class CrudTest {
>>>> >
>>>> >
>>>> >        private static final String KEYSPACE = "Keyspace1";
>>>> >
>>>> >
>>>> >        public static void main(String[] args) {
>>>> >                CrudTest client = new CrudTest();
>>>> >
>>>> >                try {
>>>> >                        client.run();
>>>> >                } catch (Exception e) {
>>>> >                        e.printStackTrace();
>>>> >                }
>>>> >
>>>> >        }
>>>> >
>>>> >
>>>> >        public void run() throws TException,
>>>> > InvalidRequestException, UnavailableException,
>>>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>>>> > {
>>>> >                TTransport tr = new TSocket("localhost", 9160);
>>>> >                TProtocol proto = new TBinaryProtocol(tr);
>>>> >                Cassandra.Client client = new
>>>> > Cassandra.Client(proto);
>>>> >                tr.open();
>>>> >
>>>> >                System.out.println("******** CREATING DATA
>>>> > *********");
>>>> >                createData(client);
>>>> >                getData(client);
>>>> >                System.out.println();
>>>> >                System.out.println("******** DELETING DATA
>>>> > *********");
>>>> >                deleteData(client);
>>>> >                getData(client);
>>>> >                System.out.println();
>>>> >                System.out.println("******** CREATING DATA
>>>> > *********");
>>>> >                createData(client);
>>>> >                getData(client);
>>>> >
>>>> >                tr.close();
>>>> >          }
>>>> >
>>>> >
>>>> >        private void createData(Cassandra.Client client) throws
>>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>>> > TException {
>>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>>>> >                cp1.setSuper_column("hotel".getBytes());
>>>> >                cp1.setColumn("Best Western".getBytes());
>>>> >
>>>> >
>>>> >                client.insert(KEYSPACE,
>>>> >                          "name",
>>>> >                          cp1,
>>>> >                          "Best Western of SF".getBytes(),
>>>> >                          System.currentTimeMillis(),
>>>> >                          ConsistencyLevel.ALL);
>>>> >
>>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>>>> >                cp2.setSuper_column("hotel".getBytes());
>>>> >                cp2.setColumn("Econolodge".getBytes());
>>>> >
>>>> >                client.insert(KEYSPACE,
>>>> >                                  "name",
>>>> >                                  cp2,
>>>> >                                  "Econolodge of SF".getBytes(),
>>>> >                                  System.currentTimeMillis(),
>>>> >                                  ConsistencyLevel.ALL);
>>>> >
>>>> >        }
>>>> >
>>>> >
>>>> >        private void deleteData(Cassandra.Client client) throws
>>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>>> > TException {
>>>> >
>>>> >                client.remove(KEYSPACE,
>>>> >                                  "hotel",
>>>> >                                  new ColumnPath("Super2"),
>>>> >                                  System.currentTimeMillis(),
>>>> >                                  ConsistencyLevel.ONE);
>>>> >
>>>> >        }
>>>> >
>>>> >
>>>> >        private void getData(Cassandra.Client client) throws
>>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>>> > TException {
>>>> >                SliceRange sliceRange = new SliceRange();
>>>> >                sliceRange.setStart(new byte[] {});
>>>> >                sliceRange.setFinish(new byte[] {});
>>>> >
>>>> >                SlicePredicate slicePredicate = new
>>>> > SlicePredicate();
>>>> >                slicePredicate.setSlice_range(sliceRange);
>>>> >
>>>> >                getData(client, slicePredicate);
>>>> >        }
>>>> >
>>>> >
>>>> >        private void getData(Cassandra.Client client,
>>>> > SlicePredicate
>>>> > slicePredicate) throws InvalidRequestException,
>>>> > UnavailableException, TimedOutException, TException {
>>>> >                List<ColumnOrSuperColumn> coscList =
>>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>>>> > slicePredicate, ConsistencyLevel.ALL);
>>>> >
>>>> >                if (coscList.isEmpty()) {
>>>> >                        System.out.println("Column Or Super Column
>>>> > is EMPTY");
>>>> >                }
>>>> >
>>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>>>> >
>>>> >                        if (cosc == null) {
>>>> >                                System.out.println("NULL RETURN
>>>> > VALUE");
>>>> >                        }
>>>> >
>>>> >                        SuperColumn superColumn =
>>>> > cosc.getSuper_column();
>>>> >
>>>> >                        if (superColumn == null) {
>>>> >                                System.out.println("Super Column is
>>>> > NULL");
>>>> >                        }
>>>> >                        else {
>>>> >                                showSuperColumnInfo(superColumn);
>>>> >                        }
>>>> >
>>>> >                }
>>>> >
>>>> >        }
>>>> >
>>>> >
>>>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>>>> >                System.out.println("######## Super Columns
>>>> > ###########");
>>>> >                System.out.println("Super Column Name = " + new
>>>> > String(superColumn.getName()));
>>>> >                List<Column> columnList = superColumn.getColumns();
>>>> >
>>>> >                System.out.println("--------- Start Columns
>>>> > -----------");
>>>> >                for (Column column: columnList) {
>>>> >                        System.out.println("Column Name = " + new
>>>> > String(column.getName()));
>>>> >                        System.out.println("Column Value = " + new
>>>> > String(column.getValue()));
>>>> >                }
>>>> >                System.out.println("--------- End Columns
>>>> > -----------");
>>>> >
>>>> > System.out.println("##################################");
>>>> >        }
>>>> >
>>>> >
>>>> > }
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > -----Original Message-----
>>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>>>> > Sent: Thu 7/22/2010 1:56 PM
>>>> > To: [hidden email]
>>>> > Subject: Re: CRUD test
>>>> >
>>>> > Yes, and that was the issue. But now after I delete a row from
>>>> > cassandra-cli, I cannot insert anything back with my code. Insert
>>>> > code works does not throw any exceptions but when I read just
>>>> > inserted columns I get NotFoundException at the last line:
>>>> >
>>>> >            client = borrowClient();
>>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>>>> > CONSISTENCY_LEVEL);
>>>> >            ColumnPath cp = new ColumnPath(application);
>>>> >            cp.setSuper_column(uuid.getBytes());
>>>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>>>> >
>>>> > It makes me think that once I remove supercolumn it cannot be
>>>> > created again.
>>>> >
>>>> >
>>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:
>>>> >
>>>> > Have you checked the timestamp you're using for the subsequent
>>>> > inserts is higher than that used in the delete?
>>>> >
>>>> >
>>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>>>> > <[hidden email]>
>>>> > wrote:
>>>> >> Hi there,
>>>> >> I'm try...
>>>> > --
>>>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>>>> > chandelier wearing a boar's head.
>>>> >
>>>> > Something which you, I, and everyone else would call "Tuesday", of
>>>> > course.
>>>> >
>>>> >
>>>
>>>
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Ran Tavory
Hi Oleg, I didn't follow up the entire thread, but just to let you know that the 0.6.* version of the CLI uses microsec as the time unit for timestamps. 
Hector also uses micros to match that, however, previous versions of hector (as well as the CLI) used milliseconds, not micro. 
So if you're using hector version 0.6.0-11 or earlier, or by any chance in some other ways are mixing milisec in your app (are you using System.currentTimeMili() somewhere?) then the behavior you're seeing is expected.


On Sat, Jul 24, 2010 at 1:06 AM, Jonathan Shook <[hidden email]> wrote:
I think you are getting it.

As far as what means what at which level, it's really about using them
consistently in every case. The [row] key (or [row] key range) is a
top-level argument for all of the operations, since it is the "key" to
mapping the set of responsible nodes. The key is the part of the name
of any column which most affects how the load is apportioned in the
cluster, so it is used very early in request processing.


On Fri, Jul 23, 2010 at 4:22 PM, Peter Minearo
<[hidden email]> wrote:
> Consequentially the remove should look like:
>
>
> ColumnPath cp1 = new ColumnPath("Super2");
>                cp1.setSuper_column("Best Western".getBytes());
>
>                client.remove(KEYSPACE,
>                                  "hotel",
>                                  cp1,
>                                  System.currentTimeMillis(),
>                                  ConsistencyLevel.ONE);
>
>                ColumnPath cp2 = new ColumnPath("Super2");
>                cp2.setSuper_column("Econolodge".getBytes());
>
>                client.remove(KEYSPACE,
>                                  "hotel",
>                                  cp2,
>                                  System.currentTimeMillis(),
>                                  ConsistencyLevel.ONE);
>
>
> -----Original Message-----
> From: Peter Minearo [mailto:[hidden email]]
> Sent: Fri 7/23/2010 2:17 PM
> To: [hidden email]
> Subject: RE: CRUD test
>
> CORRECTION:
>
> ColumnPath cp1 = new ColumnPath("Super2");
> cp1.setSuper_column("Best Western".getBytes());
> cp1.setColumn("name".getBytes());
> client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);
>
>
> -----Original Message-----
> From: Peter Minearo [mailto:[hidden email]]
> Sent: Friday, July 23, 2010 2:14 PM
> To: [hidden email]
> Subject: RE: CRUD test
>
> Interesting!! Let me rephrase to make sure I understood what is going on:
>
> When Inserting data via the "insert" function/method:
>
> void insert(string keyspace, string key, ColumnPath column_path, binary value, i64 timestamp, ConsistencyLevel consistency_level)
>
> The "key" parameter is the actual Key to the "Row", which contains SuperColumns.  The 'ColumnPath' gives the path within the Key.
>
>
>
> INCORRECT:
> ColumnPath cp1 = new ColumnPath("Super2"); cp1.setSuper_column("hotel".getBytes());
> cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE, "name", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);
>
>
> CORRECT:
> ColumnPath cp1 = new ColumnPath("Super2"); cp1.setSuper_column("name".getBytes());
> cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(), ConsistencyLevel.ALL);
>
>
>
>
>
> -----Original Message-----
> From: Jonathan Shook [mailto:[hidden email]]
> Sent: Friday, July 23, 2010 1:49 PM
> To: [hidden email]
> Subject: Re: CRUD test
>
> Correct.
>
> After the initial insert,
>
> cassandra> get Keyspace1.Super2['name']
> => (super_column=hotel,
>     (column=Best Western, value=Best Western of SF, timestamp=1279916772571)
>     (column=Econolodge, value=Econolodge of SF, timestamp=1279916772573)) Returned 1 results.
>
> ... and ...
>
> cassandra> get Keyspace1.Super2['hotel']
> Returned 0 results.
>
>
>
> On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo <[hidden email]> wrote:
>> The Model Should look like:
>>
>>
>> Super2 = {
>>        hotel: {
>>                        Best Western: {name: "Best Western of SF"}
>>                        Econolodge: {name: "Econolodge of SF"}
>>        }
>> }
>>
>> Are the CRUD Operations not referencing this correctly?
>>
>>
>>
>> -----Original Message-----
>> From: Jonathan Shook [mailto:[hidden email]]
>> Sent: Friday, July 23, 2010 1:34 PM
>> To: [hidden email]
>> Subject: Re: CRUD test
>>
>> There seem to be data consistency bugs in the test.  Are "name" and "hotel" being used in a pair-wise way?
>> Specifically, the first test is using creating one and checking for the other.
>>
>> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]> wrote:
>>> Johathan,
>>> I followed your suggestion. Unfortunately, CRUD test still does not
>>> work for me. Can you provide a simplest CRUD test possible that works?
>>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]> wrote:
>>>>
>>>> I suspect that it is still your timestamps.
>>>> You can verify this with a fake timestamp generator that is simply
>>>> incremented on each getTimestamp().
>>>>
>>>> 1 millisecond is a long time for code that is wrapped tightly in a
>>>> test. You are likely using the same logical time stamp for multiple
>>>> operations.
>>>>
>>>>
>>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>>>> <[hidden email]> wrote:
>>>> > I am able to reproduce his problem. If you take the default
>>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>>>> > the code below.  You will see that the data is not getting created
>>>> > once you run the delete.  It seems to not allow you to create data
>>>> > via Thrift.  HOWEVER, data can be created via the command line tool.
>>>> >
>>>> > import java.io.UnsupportedEncodingException;
>>>> > import java.util.List;
>>>> >
>>>> > import org.apache.cassandra.thrift.Cassandra;
>>>> > import org.apache.cassandra.thrift.Column;
>>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>>>> > import org.apache.cassandra.thrift.ColumnParent;
>>>> > import org.apache.cassandra.thrift.ColumnPath;
>>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>>>> > import org.apache.cassandra.thrift.NotFoundException;
>>>> > import org.apache.cassandra.thrift.SlicePredicate;
>>>> > import org.apache.cassandra.thrift.SliceRange;
>>>> > import org.apache.cassandra.thrift.SuperColumn;
>>>> > import org.apache.cassandra.thrift.TimedOutException;
>>>> > import org.apache.cassandra.thrift.UnavailableException;
>>>> > import org.apache.thrift.TException; import
>>>> > org.apache.thrift.protocol.TBinaryProtocol;
>>>> > import org.apache.thrift.protocol.TProtocol;
>>>> > import org.apache.thrift.transport.TSocket;
>>>> > import org.apache.thrift.transport.TTransport;
>>>> >
>>>> >
>>>> > public class CrudTest {
>>>> >
>>>> >
>>>> >        private static final String KEYSPACE = "Keyspace1";
>>>> >
>>>> >
>>>> >        public static void main(String[] args) {
>>>> >                CrudTest client = new CrudTest();
>>>> >
>>>> >                try {
>>>> >                        client.run();
>>>> >                } catch (Exception e) {
>>>> >                        e.printStackTrace();
>>>> >                }
>>>> >
>>>> >        }
>>>> >
>>>> >
>>>> >        public void run() throws TException,
>>>> > InvalidRequestException, UnavailableException,
>>>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>>>> > {
>>>> >                TTransport tr = new TSocket("localhost", 9160);
>>>> >                TProtocol proto = new TBinaryProtocol(tr);
>>>> >                Cassandra.Client client = new
>>>> > Cassandra.Client(proto);
>>>> >                tr.open();
>>>> >
>>>> >                System.out.println("******** CREATING DATA
>>>> > *********");
>>>> >                createData(client);
>>>> >                getData(client);
>>>> >                System.out.println();
>>>> >                System.out.println("******** DELETING DATA
>>>> > *********");
>>>> >                deleteData(client);
>>>> >                getData(client);
>>>> >                System.out.println();
>>>> >                System.out.println("******** CREATING DATA
>>>> > *********");
>>>> >                createData(client);
>>>> >                getData(client);
>>>> >
>>>> >                tr.close();
>>>> >          }
>>>> >
>>>> >
>>>> >        private void createData(Cassandra.Client client) throws
>>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>>> > TException {
>>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>>>> >                cp1.setSuper_column("hotel".getBytes());
>>>> >                cp1.setColumn("Best Western".getBytes());
>>>> >
>>>> >
>>>> >                client.insert(KEYSPACE,
>>>> >                          "name",
>>>> >                          cp1,
>>>> >                          "Best Western of SF".getBytes(),
>>>> >                          System.currentTimeMillis(),
>>>> >                          ConsistencyLevel.ALL);
>>>> >
>>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>>>> >                cp2.setSuper_column("hotel".getBytes());
>>>> >                cp2.setColumn("Econolodge".getBytes());
>>>> >
>>>> >                client.insert(KEYSPACE,
>>>> >                                  "name",
>>>> >                                  cp2,
>>>> >                                  "Econolodge of SF".getBytes(),
>>>> >                                  System.currentTimeMillis(),
>>>> >                                  ConsistencyLevel.ALL);
>>>> >
>>>> >        }
>>>> >
>>>> >
>>>> >        private void deleteData(Cassandra.Client client) throws
>>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>>> > TException {
>>>> >
>>>> >                client.remove(KEYSPACE,
>>>> >                                  "hotel",
>>>> >                                  new ColumnPath("Super2"),
>>>> >                                  System.currentTimeMillis(),
>>>> >                                  ConsistencyLevel.ONE);
>>>> >
>>>> >        }
>>>> >
>>>> >
>>>> >        private void getData(Cassandra.Client client) throws
>>>> > InvalidRequestException, UnavailableException, TimedOutException,
>>>> > TException {
>>>> >                SliceRange sliceRange = new SliceRange();
>>>> >                sliceRange.setStart(new byte[] {});
>>>> >                sliceRange.setFinish(new byte[] {});
>>>> >
>>>> >                SlicePredicate slicePredicate = new
>>>> > SlicePredicate();
>>>> >                slicePredicate.setSlice_range(sliceRange);
>>>> >
>>>> >                getData(client, slicePredicate);
>>>> >        }
>>>> >
>>>> >
>>>> >        private void getData(Cassandra.Client client,
>>>> > SlicePredicate
>>>> > slicePredicate) throws InvalidRequestException,
>>>> > UnavailableException, TimedOutException, TException {
>>>> >                List<ColumnOrSuperColumn> coscList =
>>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>>>> > slicePredicate, ConsistencyLevel.ALL);
>>>> >
>>>> >                if (coscList.isEmpty()) {
>>>> >                        System.out.println("Column Or Super Column
>>>> > is EMPTY");
>>>> >                }
>>>> >
>>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>>>> >
>>>> >                        if (cosc == null) {
>>>> >                                System.out.println("NULL RETURN
>>>> > VALUE");
>>>> >                        }
>>>> >
>>>> >                        SuperColumn superColumn =
>>>> > cosc.getSuper_column();
>>>> >
>>>> >                        if (superColumn == null) {
>>>> >                                System.out.println("Super Column is
>>>> > NULL");
>>>> >                        }
>>>> >                        else {
>>>> >                                showSuperColumnInfo(superColumn);
>>>> >                        }
>>>> >
>>>> >                }
>>>> >
>>>> >        }
>>>> >
>>>> >
>>>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>>>> >                System.out.println("######## Super Columns
>>>> > ###########");
>>>> >                System.out.println("Super Column Name = " + new
>>>> > String(superColumn.getName()));
>>>> >                List<Column> columnList = superColumn.getColumns();
>>>> >
>>>> >                System.out.println("--------- Start Columns
>>>> > -----------");
>>>> >                for (Column column: columnList) {
>>>> >                        System.out.println("Column Name = " + new
>>>> > String(column.getName()));
>>>> >                        System.out.println("Column Value = " + new
>>>> > String(column.getValue()));
>>>> >                }
>>>> >                System.out.println("--------- End Columns
>>>> > -----------");
>>>> >
>>>> > System.out.println("##################################");
>>>> >        }
>>>> >
>>>> >
>>>> > }
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > -----Original Message-----
>>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>>>> > Sent: Thu 7/22/2010 1:56 PM
>>>> > To: [hidden email]
>>>> > Subject: Re: CRUD test
>>>> >
>>>> > Yes, and that was the issue. But now after I delete a row from
>>>> > cassandra-cli, I cannot insert anything back with my code. Insert
>>>> > code works does not throw any exceptions but when I read just
>>>> > inserted columns I get NotFoundException at the last line:
>>>> >
>>>> >            client = borrowClient();
>>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>>>> > CONSISTENCY_LEVEL);
>>>> >            ColumnPath cp = new ColumnPath(application);
>>>> >            cp.setSuper_column(uuid.getBytes());
>>>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>>>> >
>>>> > It makes me think that once I remove supercolumn it cannot be
>>>> > created again.
>>>> >
>>>> >
>>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]> wrote:
>>>> >
>>>> > Have you checked the timestamp you're using for the subsequent
>>>> > inserts is higher than that used in the delete?
>>>> >
>>>> >
>>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>>>> > <[hidden email]>
>>>> > wrote:
>>>> >> Hi there,
>>>> >> I'm try...
>>>> > --
>>>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>>>> > chandelier wearing a boar's head.
>>>> >
>>>> > Something which you, I, and everyone else would call "Tuesday", of
>>>> > course.
>>>> >
>>>> >
>>>
>>>
>>
>
>

Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Jonathan Shook
Just to clarify, microseconds may be used, but they provide the same
behavior as milliseconds if they aren't using a higher time resolution
underneath. In some cases, the microseconds are generated simply as
milliseconds * 1000, which doesn't actually fix any sequencing bugs.

On Sat, Jul 24, 2010 at 3:46 PM, Ran Tavory <[hidden email]> wrote:

> Hi Oleg, I didn't follow up the entire thread, but just to let you know that
> the 0.6.* version of the CLI uses microsec as the time unit for timestamps.
> Hector also uses micros to match that, however, previous versions of hector
> (as well as the CLI) used milliseconds, not micro.
> So if you're using hector version 0.6.0-11 or earlier, or by any chance in
> some other ways are mixing milisec in your app (are you using
> System.currentTimeMili() somewhere?) then the behavior you're seeing is
> expected.
>
> On Sat, Jul 24, 2010 at 1:06 AM, Jonathan Shook <[hidden email]> wrote:
>>
>> I think you are getting it.
>>
>> As far as what means what at which level, it's really about using them
>> consistently in every case. The [row] key (or [row] key range) is a
>> top-level argument for all of the operations, since it is the "key" to
>> mapping the set of responsible nodes. The key is the part of the name
>> of any column which most affects how the load is apportioned in the
>> cluster, so it is used very early in request processing.
>>
>>
>> On Fri, Jul 23, 2010 at 4:22 PM, Peter Minearo
>> <[hidden email]> wrote:
>> > Consequentially the remove should look like:
>> >
>> >
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> >                cp1.setSuper_column("Best Western".getBytes());
>> >
>> >                client.remove(KEYSPACE,
>> >                                  "hotel",
>> >                                  cp1,
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ONE);
>> >
>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >                cp2.setSuper_column("Econolodge".getBytes());
>> >
>> >                client.remove(KEYSPACE,
>> >                                  "hotel",
>> >                                  cp2,
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ONE);
>> >
>> >
>> > -----Original Message-----
>> > From: Peter Minearo [mailto:[hidden email]]
>> > Sent: Fri 7/23/2010 2:17 PM
>> > To: [hidden email]
>> > Subject: RE: CRUD test
>> >
>> > CORRECTION:
>> >
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("Best Western".getBytes());
>> > cp1.setColumn("name".getBytes());
>> > client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(),
>> > System.currentTimeMillis(), ConsistencyLevel.ALL);
>> >
>> >
>> > -----Original Message-----
>> > From: Peter Minearo [mailto:[hidden email]]
>> > Sent: Friday, July 23, 2010 2:14 PM
>> > To: [hidden email]
>> > Subject: RE: CRUD test
>> >
>> > Interesting!! Let me rephrase to make sure I understood what is going
>> > on:
>> >
>> > When Inserting data via the "insert" function/method:
>> >
>> > void insert(string keyspace, string key, ColumnPath column_path, binary
>> > value, i64 timestamp, ConsistencyLevel consistency_level)
>> >
>> > The "key" parameter is the actual Key to the "Row", which contains
>> > SuperColumns.  The 'ColumnPath' gives the path within the Key.
>> >
>> >
>> >
>> > INCORRECT:
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("hotel".getBytes());
>> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> > "name", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(),
>> > ConsistencyLevel.ALL);
>> >
>> >
>> > CORRECT:
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("name".getBytes());
>> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> > "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(),
>> > ConsistencyLevel.ALL);
>> >
>> >
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Jonathan Shook [mailto:[hidden email]]
>> > Sent: Friday, July 23, 2010 1:49 PM
>> > To: [hidden email]
>> > Subject: Re: CRUD test
>> >
>> > Correct.
>> >
>> > After the initial insert,
>> >
>> > cassandra> get Keyspace1.Super2['name']
>> > => (super_column=hotel,
>> >     (column=Best Western, value=Best Western of SF,
>> > timestamp=1279916772571)
>> >     (column=Econolodge, value=Econolodge of SF,
>> > timestamp=1279916772573)) Returned 1 results.
>> >
>> > ... and ...
>> >
>> > cassandra> get Keyspace1.Super2['hotel']
>> > Returned 0 results.
>> >
>> >
>> >
>> > On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo
>> > <[hidden email]> wrote:
>> >> The Model Should look like:
>> >>
>> >>
>> >> Super2 = {
>> >>        hotel: {
>> >>                        Best Western: {name: "Best Western of SF"}
>> >>                        Econolodge: {name: "Econolodge of SF"}
>> >>        }
>> >> }
>> >>
>> >> Are the CRUD Operations not referencing this correctly?
>> >>
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Jonathan Shook [mailto:[hidden email]]
>> >> Sent: Friday, July 23, 2010 1:34 PM
>> >> To: [hidden email]
>> >> Subject: Re: CRUD test
>> >>
>> >> There seem to be data consistency bugs in the test.  Are "name" and
>> >> "hotel" being used in a pair-wise way?
>> >> Specifically, the first test is using creating one and checking for the
>> >> other.
>> >>
>> >> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]>
>> >> wrote:
>> >>> Johathan,
>> >>> I followed your suggestion. Unfortunately, CRUD test still does not
>> >>> work for me. Can you provide a simplest CRUD test possible that works?
>> >>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]>
>> >>> wrote:
>> >>>>
>> >>>> I suspect that it is still your timestamps.
>> >>>> You can verify this with a fake timestamp generator that is simply
>> >>>> incremented on each getTimestamp().
>> >>>>
>> >>>> 1 millisecond is a long time for code that is wrapped tightly in a
>> >>>> test. You are likely using the same logical time stamp for multiple
>> >>>> operations.
>> >>>>
>> >>>>
>> >>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>> >>>> <[hidden email]> wrote:
>> >>>> > I am able to reproduce his problem. If you take the default
>> >>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>> >>>> > the code below.  You will see that the data is not getting created
>> >>>> > once you run the delete.  It seems to not allow you to create data
>> >>>> > via Thrift.  HOWEVER, data can be created via the command line
>> >>>> > tool.
>> >>>> >
>> >>>> > import java.io.UnsupportedEncodingException;
>> >>>> > import java.util.List;
>> >>>> >
>> >>>> > import org.apache.cassandra.thrift.Cassandra;
>> >>>> > import org.apache.cassandra.thrift.Column;
>> >>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>> >>>> > import org.apache.cassandra.thrift.ColumnParent;
>> >>>> > import org.apache.cassandra.thrift.ColumnPath;
>> >>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>> >>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>> >>>> > import org.apache.cassandra.thrift.NotFoundException;
>> >>>> > import org.apache.cassandra.thrift.SlicePredicate;
>> >>>> > import org.apache.cassandra.thrift.SliceRange;
>> >>>> > import org.apache.cassandra.thrift.SuperColumn;
>> >>>> > import org.apache.cassandra.thrift.TimedOutException;
>> >>>> > import org.apache.cassandra.thrift.UnavailableException;
>> >>>> > import org.apache.thrift.TException; import
>> >>>> > org.apache.thrift.protocol.TBinaryProtocol;
>> >>>> > import org.apache.thrift.protocol.TProtocol;
>> >>>> > import org.apache.thrift.transport.TSocket;
>> >>>> > import org.apache.thrift.transport.TTransport;
>> >>>> >
>> >>>> >
>> >>>> > public class CrudTest {
>> >>>> >
>> >>>> >
>> >>>> >        private static final String KEYSPACE = "Keyspace1";
>> >>>> >
>> >>>> >
>> >>>> >        public static void main(String[] args) {
>> >>>> >                CrudTest client = new CrudTest();
>> >>>> >
>> >>>> >                try {
>> >>>> >                        client.run();
>> >>>> >                } catch (Exception e) {
>> >>>> >                        e.printStackTrace();
>> >>>> >                }
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        public void run() throws TException,
>> >>>> > InvalidRequestException, UnavailableException,
>> >>>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>> >>>> > {
>> >>>> >                TTransport tr = new TSocket("localhost", 9160);
>> >>>> >                TProtocol proto = new TBinaryProtocol(tr);
>> >>>> >                Cassandra.Client client = new
>> >>>> > Cassandra.Client(proto);
>> >>>> >                tr.open();
>> >>>> >
>> >>>> >                System.out.println("******** CREATING DATA
>> >>>> > *********");
>> >>>> >                createData(client);
>> >>>> >                getData(client);
>> >>>> >                System.out.println();
>> >>>> >                System.out.println("******** DELETING DATA
>> >>>> > *********");
>> >>>> >                deleteData(client);
>> >>>> >                getData(client);
>> >>>> >                System.out.println();
>> >>>> >                System.out.println("******** CREATING DATA
>> >>>> > *********");
>> >>>> >                createData(client);
>> >>>> >                getData(client);
>> >>>> >
>> >>>> >                tr.close();
>> >>>> >          }
>> >>>> >
>> >>>> >
>> >>>> >        private void createData(Cassandra.Client client) throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>> >>>> >                cp1.setSuper_column("hotel".getBytes());
>> >>>> >                cp1.setColumn("Best Western".getBytes());
>> >>>> >
>> >>>> >
>> >>>> >                client.insert(KEYSPACE,
>> >>>> >                          "name",
>> >>>> >                          cp1,
>> >>>> >                          "Best Western of SF".getBytes(),
>> >>>> >                          System.currentTimeMillis(),
>> >>>> >                          ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >>>> >                cp2.setSuper_column("hotel".getBytes());
>> >>>> >                cp2.setColumn("Econolodge".getBytes());
>> >>>> >
>> >>>> >                client.insert(KEYSPACE,
>> >>>> >                                  "name",
>> >>>> >                                  cp2,
>> >>>> >                                  "Econolodge of SF".getBytes(),
>> >>>> >                                  System.currentTimeMillis(),
>> >>>> >                                  ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void deleteData(Cassandra.Client client) throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >
>> >>>> >                client.remove(KEYSPACE,
>> >>>> >                                  "hotel",
>> >>>> >                                  new ColumnPath("Super2"),
>> >>>> >                                  System.currentTimeMillis(),
>> >>>> >                                  ConsistencyLevel.ONE);
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void getData(Cassandra.Client client) throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >                SliceRange sliceRange = new SliceRange();
>> >>>> >                sliceRange.setStart(new byte[] {});
>> >>>> >                sliceRange.setFinish(new byte[] {});
>> >>>> >
>> >>>> >                SlicePredicate slicePredicate = new
>> >>>> > SlicePredicate();
>> >>>> >                slicePredicate.setSlice_range(sliceRange);
>> >>>> >
>> >>>> >                getData(client, slicePredicate);
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void getData(Cassandra.Client client,
>> >>>> > SlicePredicate
>> >>>> > slicePredicate) throws InvalidRequestException,
>> >>>> > UnavailableException, TimedOutException, TException {
>> >>>> >                List<ColumnOrSuperColumn> coscList =
>> >>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>> >>>> > slicePredicate, ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >                if (coscList.isEmpty()) {
>> >>>> >                        System.out.println("Column Or Super Column
>> >>>> > is EMPTY");
>> >>>> >                }
>> >>>> >
>> >>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>> >>>> >
>> >>>> >                        if (cosc == null) {
>> >>>> >                                System.out.println("NULL RETURN
>> >>>> > VALUE");
>> >>>> >                        }
>> >>>> >
>> >>>> >                        SuperColumn superColumn =
>> >>>> > cosc.getSuper_column();
>> >>>> >
>> >>>> >                        if (superColumn == null) {
>> >>>> >                                System.out.println("Super Column is
>> >>>> > NULL");
>> >>>> >                        }
>> >>>> >                        else {
>> >>>> >                                showSuperColumnInfo(superColumn);
>> >>>> >                        }
>> >>>> >
>> >>>> >                }
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>> >>>> >                System.out.println("######## Super Columns
>> >>>> > ###########");
>> >>>> >                System.out.println("Super Column Name = " + new
>> >>>> > String(superColumn.getName()));
>> >>>> >                List<Column> columnList = superColumn.getColumns();
>> >>>> >
>> >>>> >                System.out.println("--------- Start Columns
>> >>>> > -----------");
>> >>>> >                for (Column column: columnList) {
>> >>>> >                        System.out.println("Column Name = " + new
>> >>>> > String(column.getName()));
>> >>>> >                        System.out.println("Column Value = " + new
>> >>>> > String(column.getValue()));
>> >>>> >                }
>> >>>> >                System.out.println("--------- End Columns
>> >>>> > -----------");
>> >>>> >
>> >>>> > System.out.println("##################################");
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> > }
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > -----Original Message-----
>> >>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>> >>>> > Sent: Thu 7/22/2010 1:56 PM
>> >>>> > To: [hidden email]
>> >>>> > Subject: Re: CRUD test
>> >>>> >
>> >>>> > Yes, and that was the issue. But now after I delete a row from
>> >>>> > cassandra-cli, I cannot insert anything back with my code. Insert
>> >>>> > code works does not throw any exceptions but when I read just
>> >>>> > inserted columns I get NotFoundException at the last line:
>> >>>> >
>> >>>> >            client = borrowClient();
>> >>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>> >>>> > CONSISTENCY_LEVEL);
>> >>>> >            ColumnPath cp = new ColumnPath(application);
>> >>>> >            cp.setSuper_column(uuid.getBytes());
>> >>>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>> >>>> >
>> >>>> > It makes me think that once I remove supercolumn it cannot be
>> >>>> > created again.
>> >>>> >
>> >>>> >
>> >>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]>
>> >>>> > wrote:
>> >>>> >
>> >>>> > Have you checked the timestamp you're using for the subsequent
>> >>>> > inserts is higher than that used in the delete?
>> >>>> >
>> >>>> >
>> >>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>> >>>> > <[hidden email]>
>> >>>> > wrote:
>> >>>> >> Hi there,
>> >>>> >> I'm try...
>> >>>> > --
>> >>>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>> >>>> > chandelier wearing a boar's head.
>> >>>> >
>> >>>> > Something which you, I, and everyone else would call "Tuesday", of
>> >>>> > course.
>> >>>> >
>> >>>> >
>> >>>
>> >>>
>> >>
>> >
>> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Oleg Tsvinev
Thank you guys for your help!

Yes, I am using System.currentTimeMillis() in my CRUD test. Even though I'm still using it my tests now run as expected. I do not use cassandra-cli anymore.

@Ran great job on Hector, I wish there was more documentation but I managed.

@Jonathan, what is the recommended time source? I use batch_mutation to insert and update multiple columns atomically. Do I have to use the batch_mutation for deletion, too?

On Sat, Jul 24, 2010 at 2:36 PM, Jonathan Shook <[hidden email]> wrote:
Just to clarify, microseconds may be used, but they provide the same
behavior as milliseconds if they aren't using a higher time resolution
underneath. In some cases, the microseconds are generated simply as
milliseconds * 1000, which doesn't actually fix any sequencing bugs.

On Sat, Jul 24, 2010 at 3:46 PM, Ran Tavory <[hidden email]> wrote:
> Hi Oleg, I didn't follow up the entire thread, but just to let you know that
> the 0.6.* version of the CLI uses microsec as the time unit for timestamps.
> Hector also uses micros to match that, however, previous versions of hector
> (as well as the CLI) used milliseconds, not micro.
> So if you're using hector version 0.6.0-11 or earlier, or by any chance in
> some other ways are mixing milisec in your app (are you using
> System.currentTimeMili() somewhere?) then the behavior you're seeing is
> expected.
>
> On Sat, Jul 24, 2010 at 1:06 AM, Jonathan Shook <[hidden email]> wrote:
>>
>> I think you are getting it.
>>
>> As far as what means what at which level, it's really about using them
>> consistently in every case. The [row] key (or [row] key range) is a
>> top-level argument for all of the operations, since it is the "key" to
>> mapping the set of responsible nodes. The key is the part of the name
>> of any column which most affects how the load is apportioned in the
>> cluster, so it is used very early in request processing.
>>
>>
>> On Fri, Jul 23, 2010 at 4:22 PM, Peter Minearo
>> <[hidden email]> wrote:
>> > Consequentially the remove should look like:
>> >
>> >
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> >                cp1.setSuper_column("Best Western".getBytes());
>> >
>> >                client.remove(KEYSPACE,
>> >                                  "hotel",
>> >                                  cp1,
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ONE);
>> >
>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >                cp2.setSuper_column("Econolodge".getBytes());
>> >
>> >                client.remove(KEYSPACE,
>> >                                  "hotel",
>> >                                  cp2,
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ONE);
>> >
>> >
>> > -----Original Message-----
>> > From: Peter Minearo [mailto:[hidden email]]
>> > Sent: Fri 7/23/2010 2:17 PM
>> > To: [hidden email]
>> > Subject: RE: CRUD test
>> >
>> > CORRECTION:
>> >
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("Best Western".getBytes());
>> > cp1.setColumn("name".getBytes());
>> > client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(),
>> > System.currentTimeMillis(), ConsistencyLevel.ALL);
>> >
>> >
>> > -----Original Message-----
>> > From: Peter Minearo [mailto:[hidden email]]
>> > Sent: Friday, July 23, 2010 2:14 PM
>> > To: [hidden email]
>> > Subject: RE: CRUD test
>> >
>> > Interesting!! Let me rephrase to make sure I understood what is going
>> > on:
>> >
>> > When Inserting data via the "insert" function/method:
>> >
>> > void insert(string keyspace, string key, ColumnPath column_path, binary
>> > value, i64 timestamp, ConsistencyLevel consistency_level)
>> >
>> > The "key" parameter is the actual Key to the "Row", which contains
>> > SuperColumns.  The 'ColumnPath' gives the path within the Key.
>> >
>> >
>> >
>> > INCORRECT:
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("hotel".getBytes());
>> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> > "name", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(),
>> > ConsistencyLevel.ALL);
>> >
>> >
>> > CORRECT:
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("name".getBytes());
>> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> > "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(),
>> > ConsistencyLevel.ALL);
>> >
>> >
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Jonathan Shook [mailto:[hidden email]]
>> > Sent: Friday, July 23, 2010 1:49 PM
>> > To: [hidden email]
>> > Subject: Re: CRUD test
>> >
>> > Correct.
>> >
>> > After the initial insert,
>> >
>> > cassandra> get Keyspace1.Super2['name']
>> > => (super_column=hotel,
>> >     (column=Best Western, value=Best Western of SF,
>> > timestamp=1279916772571)
>> >     (column=Econolodge, value=Econolodge of SF,
>> > timestamp=1279916772573)) Returned 1 results.
>> >
>> > ... and ...
>> >
>> > cassandra> get Keyspace1.Super2['hotel']
>> > Returned 0 results.
>> >
>> >
>> >
>> > On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo
>> > <[hidden email]> wrote:
>> >> The Model Should look like:
>> >>
>> >>
>> >> Super2 = {
>> >>        hotel: {
>> >>                        Best Western: {name: "Best Western of SF"}
>> >>                        Econolodge: {name: "Econolodge of SF"}
>> >>        }
>> >> }
>> >>
>> >> Are the CRUD Operations not referencing this correctly?
>> >>
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Jonathan Shook [mailto:[hidden email]]
>> >> Sent: Friday, July 23, 2010 1:34 PM
>> >> To: [hidden email]
>> >> Subject: Re: CRUD test
>> >>
>> >> There seem to be data consistency bugs in the test.  Are "name" and
>> >> "hotel" being used in a pair-wise way?
>> >> Specifically, the first test is using creating one and checking for the
>> >> other.
>> >>
>> >> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]>
>> >> wrote:
>> >>> Johathan,
>> >>> I followed your suggestion. Unfortunately, CRUD test still does not
>> >>> work for me. Can you provide a simplest CRUD test possible that works?
>> >>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]>
>> >>> wrote:
>> >>>>
>> >>>> I suspect that it is still your timestamps.
>> >>>> You can verify this with a fake timestamp generator that is simply
>> >>>> incremented on each getTimestamp().
>> >>>>
>> >>>> 1 millisecond is a long time for code that is wrapped tightly in a
>> >>>> test. You are likely using the same logical time stamp for multiple
>> >>>> operations.
>> >>>>
>> >>>>
>> >>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>> >>>> <[hidden email]> wrote:
>> >>>> > I am able to reproduce his problem. If you take the default
>> >>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>> >>>> > the code below.  You will see that the data is not getting created
>> >>>> > once you run the delete.  It seems to not allow you to create data
>> >>>> > via Thrift.  HOWEVER, data can be created via the command line
>> >>>> > tool.
>> >>>> >
>> >>>> > import java.io.UnsupportedEncodingException;
>> >>>> > import java.util.List;
>> >>>> >
>> >>>> > import org.apache.cassandra.thrift.Cassandra;
>> >>>> > import org.apache.cassandra.thrift.Column;
>> >>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>> >>>> > import org.apache.cassandra.thrift.ColumnParent;
>> >>>> > import org.apache.cassandra.thrift.ColumnPath;
>> >>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>> >>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>> >>>> > import org.apache.cassandra.thrift.NotFoundException;
>> >>>> > import org.apache.cassandra.thrift.SlicePredicate;
>> >>>> > import org.apache.cassandra.thrift.SliceRange;
>> >>>> > import org.apache.cassandra.thrift.SuperColumn;
>> >>>> > import org.apache.cassandra.thrift.TimedOutException;
>> >>>> > import org.apache.cassandra.thrift.UnavailableException;
>> >>>> > import org.apache.thrift.TException; import
>> >>>> > org.apache.thrift.protocol.TBinaryProtocol;
>> >>>> > import org.apache.thrift.protocol.TProtocol;
>> >>>> > import org.apache.thrift.transport.TSocket;
>> >>>> > import org.apache.thrift.transport.TTransport;
>> >>>> >
>> >>>> >
>> >>>> > public class CrudTest {
>> >>>> >
>> >>>> >
>> >>>> >        private static final String KEYSPACE = "Keyspace1";
>> >>>> >
>> >>>> >
>> >>>> >        public static void main(String[] args) {
>> >>>> >                CrudTest client = new CrudTest();
>> >>>> >
>> >>>> >                try {
>> >>>> >                        client.run();
>> >>>> >                } catch (Exception e) {
>> >>>> >                        e.printStackTrace();
>> >>>> >                }
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        public void run() throws TException,
>> >>>> > InvalidRequestException, UnavailableException,
>> >>>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>> >>>> > {
>> >>>> >                TTransport tr = new TSocket("localhost", 9160);
>> >>>> >                TProtocol proto = new TBinaryProtocol(tr);
>> >>>> >                Cassandra.Client client = new
>> >>>> > Cassandra.Client(proto);
>> >>>> >                tr.open();
>> >>>> >
>> >>>> >                System.out.println("******** CREATING DATA
>> >>>> > *********");
>> >>>> >                createData(client);
>> >>>> >                getData(client);
>> >>>> >                System.out.println();
>> >>>> >                System.out.println("******** DELETING DATA
>> >>>> > *********");
>> >>>> >                deleteData(client);
>> >>>> >                getData(client);
>> >>>> >                System.out.println();
>> >>>> >                System.out.println("******** CREATING DATA
>> >>>> > *********");
>> >>>> >                createData(client);
>> >>>> >                getData(client);
>> >>>> >
>> >>>> >                tr.close();
>> >>>> >          }
>> >>>> >
>> >>>> >
>> >>>> >        private void createData(Cassandra.Client client) throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>> >>>> >                cp1.setSuper_column("hotel".getBytes());
>> >>>> >                cp1.setColumn("Best Western".getBytes());
>> >>>> >
>> >>>> >
>> >>>> >                client.insert(KEYSPACE,
>> >>>> >                          "name",
>> >>>> >                          cp1,
>> >>>> >                          "Best Western of SF".getBytes(),
>> >>>> >                          System.currentTimeMillis(),
>> >>>> >                          ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >>>> >                cp2.setSuper_column("hotel".getBytes());
>> >>>> >                cp2.setColumn("Econolodge".getBytes());
>> >>>> >
>> >>>> >                client.insert(KEYSPACE,
>> >>>> >                                  "name",
>> >>>> >                                  cp2,
>> >>>> >                                  "Econolodge of SF".getBytes(),
>> >>>> >                                  System.currentTimeMillis(),
>> >>>> >                                  ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void deleteData(Cassandra.Client client) throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >
>> >>>> >                client.remove(KEYSPACE,
>> >>>> >                                  "hotel",
>> >>>> >                                  new ColumnPath("Super2"),
>> >>>> >                                  System.currentTimeMillis(),
>> >>>> >                                  ConsistencyLevel.ONE);
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void getData(Cassandra.Client client) throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >                SliceRange sliceRange = new SliceRange();
>> >>>> >                sliceRange.setStart(new byte[] {});
>> >>>> >                sliceRange.setFinish(new byte[] {});
>> >>>> >
>> >>>> >                SlicePredicate slicePredicate = new
>> >>>> > SlicePredicate();
>> >>>> >                slicePredicate.setSlice_range(sliceRange);
>> >>>> >
>> >>>> >                getData(client, slicePredicate);
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void getData(Cassandra.Client client,
>> >>>> > SlicePredicate
>> >>>> > slicePredicate) throws InvalidRequestException,
>> >>>> > UnavailableException, TimedOutException, TException {
>> >>>> >                List<ColumnOrSuperColumn> coscList =
>> >>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>> >>>> > slicePredicate, ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >                if (coscList.isEmpty()) {
>> >>>> >                        System.out.println("Column Or Super Column
>> >>>> > is EMPTY");
>> >>>> >                }
>> >>>> >
>> >>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>> >>>> >
>> >>>> >                        if (cosc == null) {
>> >>>> >                                System.out.println("NULL RETURN
>> >>>> > VALUE");
>> >>>> >                        }
>> >>>> >
>> >>>> >                        SuperColumn superColumn =
>> >>>> > cosc.getSuper_column();
>> >>>> >
>> >>>> >                        if (superColumn == null) {
>> >>>> >                                System.out.println("Super Column is
>> >>>> > NULL");
>> >>>> >                        }
>> >>>> >                        else {
>> >>>> >                                showSuperColumnInfo(superColumn);
>> >>>> >                        }
>> >>>> >
>> >>>> >                }
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>> >>>> >                System.out.println("######## Super Columns
>> >>>> > ###########");
>> >>>> >                System.out.println("Super Column Name = " + new
>> >>>> > String(superColumn.getName()));
>> >>>> >                List<Column> columnList = superColumn.getColumns();
>> >>>> >
>> >>>> >                System.out.println("--------- Start Columns
>> >>>> > -----------");
>> >>>> >                for (Column column: columnList) {
>> >>>> >                        System.out.println("Column Name = " + new
>> >>>> > String(column.getName()));
>> >>>> >                        System.out.println("Column Value = " + new
>> >>>> > String(column.getValue()));
>> >>>> >                }
>> >>>> >                System.out.println("--------- End Columns
>> >>>> > -----------");
>> >>>> >
>> >>>> > System.out.println("##################################");
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> > }
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > -----Original Message-----
>> >>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>> >>>> > Sent: Thu 7/22/2010 1:56 PM
>> >>>> > To: [hidden email]
>> >>>> > Subject: Re: CRUD test
>> >>>> >
>> >>>> > Yes, and that was the issue. But now after I delete a row from
>> >>>> > cassandra-cli, I cannot insert anything back with my code. Insert
>> >>>> > code works does not throw any exceptions but when I read just
>> >>>> > inserted columns I get NotFoundException at the last line:
>> >>>> >
>> >>>> >            client = borrowClient();
>> >>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>> >>>> > CONSISTENCY_LEVEL);
>> >>>> >            ColumnPath cp = new ColumnPath(application);
>> >>>> >            cp.setSuper_column(uuid.getBytes());
>> >>>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>> >>>> >
>> >>>> > It makes me think that once I remove supercolumn it cannot be
>> >>>> > created again.
>> >>>> >
>> >>>> >
>> >>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]>
>> >>>> > wrote:
>> >>>> >
>> >>>> > Have you checked the timestamp you're using for the subsequent
>> >>>> > inserts is higher than that used in the delete?
>> >>>> >
>> >>>> >
>> >>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>> >>>> > <[hidden email]>
>> >>>> > wrote:
>> >>>> >> Hi there,
>> >>>> >> I'm try...
>> >>>> > --
>> >>>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>> >>>> > chandelier wearing a boar's head.
>> >>>> >
>> >>>> > Something which you, I, and everyone else would call "Tuesday", of
>> >>>> > course.
>> >>>> >
>> >>>> >
>> >>>
>> >>>
>> >>
>> >
>> >
>
>

Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Ran Tavory
Oleg, note that the unofficial recommendation is to use microsec, not mili. As jonathan notes, although there isn't a real way to get microsec in java, at the very least you should take the mili and multiply it by 1000. If you use hector then just use Keyspace.createTimestamp() (http://github.com/rantav/hector/blob/master/src/main/java/me/prettyprint/cassandra/service/Keyspace.java#L236)

On Sun, Jul 25, 2010 at 8:54 AM, Oleg Tsvinev <[hidden email]> wrote:
Thank you guys for your help!

Yes, I am using System.currentTimeMillis() in my CRUD test. Even though I'm still using it my tests now run as expected. I do not use cassandra-cli anymore.

@Ran great job on Hector, I wish there was more documentation but I managed.

@Jonathan, what is the recommended time source? I use batch_mutation to insert and update multiple columns atomically. Do I have to use the batch_mutation for deletion, too?

On Sat, Jul 24, 2010 at 2:36 PM, Jonathan Shook <[hidden email]> wrote:
Just to clarify, microseconds may be used, but they provide the same
behavior as milliseconds if they aren't using a higher time resolution
underneath. In some cases, the microseconds are generated simply as
milliseconds * 1000, which doesn't actually fix any sequencing bugs.

On Sat, Jul 24, 2010 at 3:46 PM, Ran Tavory <[hidden email]> wrote:
> Hi Oleg, I didn't follow up the entire thread, but just to let you know that
> the 0.6.* version of the CLI uses microsec as the time unit for timestamps.
> Hector also uses micros to match that, however, previous versions of hector
> (as well as the CLI) used milliseconds, not micro.
> So if you're using hector version 0.6.0-11 or earlier, or by any chance in
> some other ways are mixing milisec in your app (are you using
> System.currentTimeMili() somewhere?) then the behavior you're seeing is
> expected.
>
> On Sat, Jul 24, 2010 at 1:06 AM, Jonathan Shook <[hidden email]> wrote:
>>
>> I think you are getting it.
>>
>> As far as what means what at which level, it's really about using them
>> consistently in every case. The [row] key (or [row] key range) is a
>> top-level argument for all of the operations, since it is the "key" to
>> mapping the set of responsible nodes. The key is the part of the name
>> of any column which most affects how the load is apportioned in the
>> cluster, so it is used very early in request processing.
>>
>>
>> On Fri, Jul 23, 2010 at 4:22 PM, Peter Minearo
>> <[hidden email]> wrote:
>> > Consequentially the remove should look like:
>> >
>> >
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> >                cp1.setSuper_column("Best Western".getBytes());
>> >
>> >                client.remove(KEYSPACE,
>> >                                  "hotel",
>> >                                  cp1,
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ONE);
>> >
>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >                cp2.setSuper_column("Econolodge".getBytes());
>> >
>> >                client.remove(KEYSPACE,
>> >                                  "hotel",
>> >                                  cp2,
>> >                                  System.currentTimeMillis(),
>> >                                  ConsistencyLevel.ONE);
>> >
>> >
>> > -----Original Message-----
>> > From: Peter Minearo [mailto:[hidden email]]
>> > Sent: Fri 7/23/2010 2:17 PM
>> > To: [hidden email]
>> > Subject: RE: CRUD test
>> >
>> > CORRECTION:
>> >
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("Best Western".getBytes());
>> > cp1.setColumn("name".getBytes());
>> > client.insert(KEYSPACE, "hotel", cp1, "Best Western of SF".getBytes(),
>> > System.currentTimeMillis(), ConsistencyLevel.ALL);
>> >
>> >
>> > -----Original Message-----
>> > From: Peter Minearo [mailto:[hidden email]]
>> > Sent: Friday, July 23, 2010 2:14 PM
>> > To: [hidden email]
>> > Subject: RE: CRUD test
>> >
>> > Interesting!! Let me rephrase to make sure I understood what is going
>> > on:
>> >
>> > When Inserting data via the "insert" function/method:
>> >
>> > void insert(string keyspace, string key, ColumnPath column_path, binary
>> > value, i64 timestamp, ConsistencyLevel consistency_level)
>> >
>> > The "key" parameter is the actual Key to the "Row", which contains
>> > SuperColumns.  The 'ColumnPath' gives the path within the Key.
>> >
>> >
>> >
>> > INCORRECT:
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("hotel".getBytes());
>> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> > "name", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(),
>> > ConsistencyLevel.ALL);
>> >
>> >
>> > CORRECT:
>> > ColumnPath cp1 = new ColumnPath("Super2");
>> > cp1.setSuper_column("name".getBytes());
>> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> > "hotel", cp1, "Best Western of SF".getBytes(), System.currentTimeMillis(),
>> > ConsistencyLevel.ALL);
>> >
>> >
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Jonathan Shook [mailto:[hidden email]]
>> > Sent: Friday, July 23, 2010 1:49 PM
>> > To: [hidden email]
>> > Subject: Re: CRUD test
>> >
>> > Correct.
>> >
>> > After the initial insert,
>> >
>> > cassandra> get Keyspace1.Super2['name']
>> > => (super_column=hotel,
>> >     (column=Best Western, value=Best Western of SF,
>> > timestamp=1279916772571)
>> >     (column=Econolodge, value=Econolodge of SF,
>> > timestamp=1279916772573)) Returned 1 results.
>> >
>> > ... and ...
>> >
>> > cassandra> get Keyspace1.Super2['hotel']
>> > Returned 0 results.
>> >
>> >
>> >
>> > On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo
>> > <[hidden email]> wrote:
>> >> The Model Should look like:
>> >>
>> >>
>> >> Super2 = {
>> >>        hotel: {
>> >>                        Best Western: {name: "Best Western of SF"}
>> >>                        Econolodge: {name: "Econolodge of SF"}
>> >>        }
>> >> }
>> >>
>> >> Are the CRUD Operations not referencing this correctly?
>> >>
>> >>
>> >>
>> >> -----Original Message-----
>> >> From: Jonathan Shook [mailto:[hidden email]]
>> >> Sent: Friday, July 23, 2010 1:34 PM
>> >> To: [hidden email]
>> >> Subject: Re: CRUD test
>> >>
>> >> There seem to be data consistency bugs in the test.  Are "name" and
>> >> "hotel" being used in a pair-wise way?
>> >> Specifically, the first test is using creating one and checking for the
>> >> other.
>> >>
>> >> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev <[hidden email]>
>> >> wrote:
>> >>> Johathan,
>> >>> I followed your suggestion. Unfortunately, CRUD test still does not
>> >>> work for me. Can you provide a simplest CRUD test possible that works?
>> >>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]>
>> >>> wrote:
>> >>>>
>> >>>> I suspect that it is still your timestamps.
>> >>>> You can verify this with a fake timestamp generator that is simply
>> >>>> incremented on each getTimestamp().
>> >>>>
>> >>>> 1 millisecond is a long time for code that is wrapped tightly in a
>> >>>> test. You are likely using the same logical time stamp for multiple
>> >>>> operations.
>> >>>>
>> >>>>
>> >>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>> >>>> <[hidden email]> wrote:
>> >>>> > I am able to reproduce his problem. If you take the default
>> >>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>> >>>> > the code below.  You will see that the data is not getting created
>> >>>> > once you run the delete.  It seems to not allow you to create data
>> >>>> > via Thrift.  HOWEVER, data can be created via the command line
>> >>>> > tool.
>> >>>> >
>> >>>> > import java.io.UnsupportedEncodingException;
>> >>>> > import java.util.List;
>> >>>> >
>> >>>> > import org.apache.cassandra.thrift.Cassandra;
>> >>>> > import org.apache.cassandra.thrift.Column;
>> >>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>> >>>> > import org.apache.cassandra.thrift.ColumnParent;
>> >>>> > import org.apache.cassandra.thrift.ColumnPath;
>> >>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>> >>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>> >>>> > import org.apache.cassandra.thrift.NotFoundException;
>> >>>> > import org.apache.cassandra.thrift.SlicePredicate;
>> >>>> > import org.apache.cassandra.thrift.SliceRange;
>> >>>> > import org.apache.cassandra.thrift.SuperColumn;
>> >>>> > import org.apache.cassandra.thrift.TimedOutException;
>> >>>> > import org.apache.cassandra.thrift.UnavailableException;
>> >>>> > import org.apache.thrift.TException; import
>> >>>> > org.apache.thrift.protocol.TBinaryProtocol;
>> >>>> > import org.apache.thrift.protocol.TProtocol;
>> >>>> > import org.apache.thrift.transport.TSocket;
>> >>>> > import org.apache.thrift.transport.TTransport;
>> >>>> >
>> >>>> >
>> >>>> > public class CrudTest {
>> >>>> >
>> >>>> >
>> >>>> >        private static final String KEYSPACE = "Keyspace1";
>> >>>> >
>> >>>> >
>> >>>> >        public static void main(String[] args) {
>> >>>> >                CrudTest client = new CrudTest();
>> >>>> >
>> >>>> >                try {
>> >>>> >                        client.run();
>> >>>> >                } catch (Exception e) {
>> >>>> >                        e.printStackTrace();
>> >>>> >                }
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        public void run() throws TException,
>> >>>> > InvalidRequestException, UnavailableException,
>> >>>> > UnsupportedEncodingException, NotFoundException, TimedOutException
>> >>>> > {
>> >>>> >                TTransport tr = new TSocket("localhost", 9160);
>> >>>> >                TProtocol proto = new TBinaryProtocol(tr);
>> >>>> >                Cassandra.Client client = new
>> >>>> > Cassandra.Client(proto);
>> >>>> >                tr.open();
>> >>>> >
>> >>>> >                System.out.println("******** CREATING DATA
>> >>>> > *********");
>> >>>> >                createData(client);
>> >>>> >                getData(client);
>> >>>> >                System.out.println();
>> >>>> >                System.out.println("******** DELETING DATA
>> >>>> > *********");
>> >>>> >                deleteData(client);
>> >>>> >                getData(client);
>> >>>> >                System.out.println();
>> >>>> >                System.out.println("******** CREATING DATA
>> >>>> > *********");
>> >>>> >                createData(client);
>> >>>> >                getData(client);
>> >>>> >
>> >>>> >                tr.close();
>> >>>> >          }
>> >>>> >
>> >>>> >
>> >>>> >        private void createData(Cassandra.Client client) throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>> >>>> >                cp1.setSuper_column("hotel".getBytes());
>> >>>> >                cp1.setColumn("Best Western".getBytes());
>> >>>> >
>> >>>> >
>> >>>> >                client.insert(KEYSPACE,
>> >>>> >                          "name",
>> >>>> >                          cp1,
>> >>>> >                          "Best Western of SF".getBytes(),
>> >>>> >                          System.currentTimeMillis(),
>> >>>> >                          ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >>>> >                cp2.setSuper_column("hotel".getBytes());
>> >>>> >                cp2.setColumn("Econolodge".getBytes());
>> >>>> >
>> >>>> >                client.insert(KEYSPACE,
>> >>>> >                                  "name",
>> >>>> >                                  cp2,
>> >>>> >                                  "Econolodge of SF".getBytes(),
>> >>>> >                                  System.currentTimeMillis(),
>> >>>> >                                  ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void deleteData(Cassandra.Client client) throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >
>> >>>> >                client.remove(KEYSPACE,
>> >>>> >                                  "hotel",
>> >>>> >                                  new ColumnPath("Super2"),
>> >>>> >                                  System.currentTimeMillis(),
>> >>>> >                                  ConsistencyLevel.ONE);
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void getData(Cassandra.Client client) throws
>> >>>> > InvalidRequestException, UnavailableException, TimedOutException,
>> >>>> > TException {
>> >>>> >                SliceRange sliceRange = new SliceRange();
>> >>>> >                sliceRange.setStart(new byte[] {});
>> >>>> >                sliceRange.setFinish(new byte[] {});
>> >>>> >
>> >>>> >                SlicePredicate slicePredicate = new
>> >>>> > SlicePredicate();
>> >>>> >                slicePredicate.setSlice_range(sliceRange);
>> >>>> >
>> >>>> >                getData(client, slicePredicate);
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void getData(Cassandra.Client client,
>> >>>> > SlicePredicate
>> >>>> > slicePredicate) throws InvalidRequestException,
>> >>>> > UnavailableException, TimedOutException, TException {
>> >>>> >                List<ColumnOrSuperColumn> coscList =
>> >>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>> >>>> > slicePredicate, ConsistencyLevel.ALL);
>> >>>> >
>> >>>> >                if (coscList.isEmpty()) {
>> >>>> >                        System.out.println("Column Or Super Column
>> >>>> > is EMPTY");
>> >>>> >                }
>> >>>> >
>> >>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>> >>>> >
>> >>>> >                        if (cosc == null) {
>> >>>> >                                System.out.println("NULL RETURN
>> >>>> > VALUE");
>> >>>> >                        }
>> >>>> >
>> >>>> >                        SuperColumn superColumn =
>> >>>> > cosc.getSuper_column();
>> >>>> >
>> >>>> >                        if (superColumn == null) {
>> >>>> >                                System.out.println("Super Column is
>> >>>> > NULL");
>> >>>> >                        }
>> >>>> >                        else {
>> >>>> >                                showSuperColumnInfo(superColumn);
>> >>>> >                        }
>> >>>> >
>> >>>> >                }
>> >>>> >
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> >        private void showSuperColumnInfo(SuperColumn superColumn) {
>> >>>> >                System.out.println("######## Super Columns
>> >>>> > ###########");
>> >>>> >                System.out.println("Super Column Name = " + new
>> >>>> > String(superColumn.getName()));
>> >>>> >                List<Column> columnList = superColumn.getColumns();
>> >>>> >
>> >>>> >                System.out.println("--------- Start Columns
>> >>>> > -----------");
>> >>>> >                for (Column column: columnList) {
>> >>>> >                        System.out.println("Column Name = " + new
>> >>>> > String(column.getName()));
>> >>>> >                        System.out.println("Column Value = " + new
>> >>>> > String(column.getValue()));
>> >>>> >                }
>> >>>> >                System.out.println("--------- End Columns
>> >>>> > -----------");
>> >>>> >
>> >>>> > System.out.println("##################################");
>> >>>> >        }
>> >>>> >
>> >>>> >
>> >>>> > }
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> >
>> >>>> > -----Original Message-----
>> >>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>> >>>> > Sent: Thu 7/22/2010 1:56 PM
>> >>>> > To: [hidden email]
>> >>>> > Subject: Re: CRUD test
>> >>>> >
>> >>>> > Yes, and that was the issue. But now after I delete a row from
>> >>>> > cassandra-cli, I cannot insert anything back with my code. Insert
>> >>>> > code works does not throw any exceptions but when I read just
>> >>>> > inserted columns I get NotFoundException at the last line:
>> >>>> >
>> >>>> >            client = borrowClient();
>> >>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>> >>>> > CONSISTENCY_LEVEL);
>> >>>> >            ColumnPath cp = new ColumnPath(application);
>> >>>> >            cp.setSuper_column(uuid.getBytes());
>> >>>> >            SuperColumn sc = keyspace.getSuperColumn(category, cp);
>> >>>> >
>> >>>> > It makes me think that once I remove supercolumn it cannot be
>> >>>> > created again.
>> >>>> >
>> >>>> >
>> >>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]>
>> >>>> > wrote:
>> >>>> >
>> >>>> > Have you checked the timestamp you're using for the subsequent
>> >>>> > inserts is higher than that used in the delete?
>> >>>> >
>> >>>> >
>> >>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>> >>>> > <[hidden email]>
>> >>>> > wrote:
>> >>>> >> Hi there,
>> >>>> >> I'm try...
>> >>>> > --
>> >>>> > Maybe she awoke to see the roommate's boyfriend swinging from the
>> >>>> > chandelier wearing a boar's head.
>> >>>> >
>> >>>> > Something which you, I, and everyone else would call "Tuesday", of
>> >>>> > course.
>> >>>> >
>> >>>> >
>> >>>
>> >>>
>> >>
>> >
>> >
>
>


Reply | Threaded
Open this post in threaded view
|

Re: CRUD test

Jonathan Shook
In reply to this post by Oleg Tsvinev
That's a question that many Java developers would like the answer to.
Unfortunately, anything better than milliseconds requires JNI, since
the current JVM doesn't officially support anything higher. There are
solutions to this particular problem, but for most people,
milliseconds are sufficient outside of testing. This is because the
likelihood of making two conflicting changes to the same row/column in
the same "session" within the same millisecond is pretty low for
actual users and real scenarios. Tests tend to be a little unrealistic
in the sense that they happen quickly within a short amount of time
and aren't dependent on the timing of people or other systems.

If you are using a "remove and replace" scheme, it could still be a
problem. The way I get around it for now is to use the microsecond
unit of time with a millisecond source (getCurrentMillis()), and
increment it artificially when it would return the same value twice in
a row. It's a hack, but it works for my purposes.

On Sun, Jul 25, 2010 at 12:54 AM, Oleg Tsvinev <[hidden email]> wrote:

> Thank you guys for your help!
> Yes, I am using System.currentTimeMillis() in my CRUD test. Even though I'm
> still using it my tests now run as expected. I do not use cassandra-cli
> anymore.
> @Ran great job on Hector, I wish there was more documentation but I managed.
> @Jonathan, what is the recommended time source? I use batch_mutation to
> insert and update multiple columns atomically. Do I have to use
> the batch_mutation for deletion, too?
> On Sat, Jul 24, 2010 at 2:36 PM, Jonathan Shook <[hidden email]> wrote:
>>
>> Just to clarify, microseconds may be used, but they provide the same
>> behavior as milliseconds if they aren't using a higher time resolution
>> underneath. In some cases, the microseconds are generated simply as
>> milliseconds * 1000, which doesn't actually fix any sequencing bugs.
>>
>> On Sat, Jul 24, 2010 at 3:46 PM, Ran Tavory <[hidden email]> wrote:
>> > Hi Oleg, I didn't follow up the entire thread, but just to let you know
>> > that
>> > the 0.6.* version of the CLI uses microsec as the time unit for
>> > timestamps.
>> > Hector also uses micros to match that, however, previous versions of
>> > hector
>> > (as well as the CLI) used milliseconds, not micro.
>> > So if you're using hector version 0.6.0-11 or earlier, or by any chance
>> > in
>> > some other ways are mixing milisec in your app (are you using
>> > System.currentTimeMili() somewhere?) then the behavior you're seeing is
>> > expected.
>> >
>> > On Sat, Jul 24, 2010 at 1:06 AM, Jonathan Shook <[hidden email]>
>> > wrote:
>> >>
>> >> I think you are getting it.
>> >>
>> >> As far as what means what at which level, it's really about using them
>> >> consistently in every case. The [row] key (or [row] key range) is a
>> >> top-level argument for all of the operations, since it is the "key" to
>> >> mapping the set of responsible nodes. The key is the part of the name
>> >> of any column which most affects how the load is apportioned in the
>> >> cluster, so it is used very early in request processing.
>> >>
>> >>
>> >> On Fri, Jul 23, 2010 at 4:22 PM, Peter Minearo
>> >> <[hidden email]> wrote:
>> >> > Consequentially the remove should look like:
>> >> >
>> >> >
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> >                cp1.setSuper_column("Best Western".getBytes());
>> >> >
>> >> >                client.remove(KEYSPACE,
>> >> >                                  "hotel",
>> >> >                                  cp1,
>> >> >                                  System.currentTimeMillis(),
>> >> >                                  ConsistencyLevel.ONE);
>> >> >
>> >> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >> >                cp2.setSuper_column("Econolodge".getBytes());
>> >> >
>> >> >                client.remove(KEYSPACE,
>> >> >                                  "hotel",
>> >> >                                  cp2,
>> >> >                                  System.currentTimeMillis(),
>> >> >                                  ConsistencyLevel.ONE);
>> >> >
>> >> >
>> >> > -----Original Message-----
>> >> > From: Peter Minearo [mailto:[hidden email]]
>> >> > Sent: Fri 7/23/2010 2:17 PM
>> >> > To: [hidden email]
>> >> > Subject: RE: CRUD test
>> >> >
>> >> > CORRECTION:
>> >> >
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> > cp1.setSuper_column("Best Western".getBytes());
>> >> > cp1.setColumn("name".getBytes());
>> >> > client.insert(KEYSPACE, "hotel", cp1, "Best Western of
>> >> > SF".getBytes(),
>> >> > System.currentTimeMillis(), ConsistencyLevel.ALL);
>> >> >
>> >> >
>> >> > -----Original Message-----
>> >> > From: Peter Minearo [mailto:[hidden email]]
>> >> > Sent: Friday, July 23, 2010 2:14 PM
>> >> > To: [hidden email]
>> >> > Subject: RE: CRUD test
>> >> >
>> >> > Interesting!! Let me rephrase to make sure I understood what is going
>> >> > on:
>> >> >
>> >> > When Inserting data via the "insert" function/method:
>> >> >
>> >> > void insert(string keyspace, string key, ColumnPath column_path,
>> >> > binary
>> >> > value, i64 timestamp, ConsistencyLevel consistency_level)
>> >> >
>> >> > The "key" parameter is the actual Key to the "Row", which contains
>> >> > SuperColumns.  The 'ColumnPath' gives the path within the Key.
>> >> >
>> >> >
>> >> >
>> >> > INCORRECT:
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> > cp1.setSuper_column("hotel".getBytes());
>> >> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> >> > "name", cp1, "Best Western of SF".getBytes(),
>> >> > System.currentTimeMillis(),
>> >> > ConsistencyLevel.ALL);
>> >> >
>> >> >
>> >> > CORRECT:
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> > cp1.setSuper_column("name".getBytes());
>> >> > cp1.setColumn("Best Western".getBytes()); client.insert(KEYSPACE,
>> >> > "hotel", cp1, "Best Western of SF".getBytes(),
>> >> > System.currentTimeMillis(),
>> >> > ConsistencyLevel.ALL);
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > -----Original Message-----
>> >> > From: Jonathan Shook [mailto:[hidden email]]
>> >> > Sent: Friday, July 23, 2010 1:49 PM
>> >> > To: [hidden email]
>> >> > Subject: Re: CRUD test
>> >> >
>> >> > Correct.
>> >> >
>> >> > After the initial insert,
>> >> >
>> >> > cassandra> get Keyspace1.Super2['name']
>> >> > => (super_column=hotel,
>> >> >     (column=Best Western, value=Best Western of SF,
>> >> > timestamp=1279916772571)
>> >> >     (column=Econolodge, value=Econolodge of SF,
>> >> > timestamp=1279916772573)) Returned 1 results.
>> >> >
>> >> > ... and ...
>> >> >
>> >> > cassandra> get Keyspace1.Super2['hotel']
>> >> > Returned 0 results.
>> >> >
>> >> >
>> >> >
>> >> > On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo
>> >> > <[hidden email]> wrote:
>> >> >> The Model Should look like:
>> >> >>
>> >> >>
>> >> >> Super2 = {
>> >> >>        hotel: {
>> >> >>                        Best Western: {name: "Best Western of SF"}
>> >> >>                        Econolodge: {name: "Econolodge of SF"}
>> >> >>        }
>> >> >> }
>> >> >>
>> >> >> Are the CRUD Operations not referencing this correctly?
>> >> >>
>> >> >>
>> >> >>
>> >> >> -----Original Message-----
>> >> >> From: Jonathan Shook [mailto:[hidden email]]
>> >> >> Sent: Friday, July 23, 2010 1:34 PM
>> >> >> To: [hidden email]
>> >> >> Subject: Re: CRUD test
>> >> >>
>> >> >> There seem to be data consistency bugs in the test.  Are "name" and
>> >> >> "hotel" being used in a pair-wise way?
>> >> >> Specifically, the first test is using creating one and checking for
>> >> >> the
>> >> >> other.
>> >> >>
>> >> >> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev
>> >> >> <[hidden email]>
>> >> >> wrote:
>> >> >>> Johathan,
>> >> >>> I followed your suggestion. Unfortunately, CRUD test still does not
>> >> >>> work for me. Can you provide a simplest CRUD test possible that
>> >> >>> works?
>> >> >>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook <[hidden email]>
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> I suspect that it is still your timestamps.
>> >> >>>> You can verify this with a fake timestamp generator that is simply
>> >> >>>> incremented on each getTimestamp().
>> >> >>>>
>> >> >>>> 1 millisecond is a long time for code that is wrapped tightly in a
>> >> >>>> test. You are likely using the same logical time stamp for
>> >> >>>> multiple
>> >> >>>> operations.
>> >> >>>>
>> >> >>>>
>> >> >>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>> >> >>>> <[hidden email]> wrote:
>> >> >>>> > I am able to reproduce his problem. If you take the default
>> >> >>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily with
>> >> >>>> > the code below.  You will see that the data is not getting
>> >> >>>> > created
>> >> >>>> > once you run the delete.  It seems to not allow you to create
>> >> >>>> > data
>> >> >>>> > via Thrift.  HOWEVER, data can be created via the command line
>> >> >>>> > tool.
>> >> >>>> >
>> >> >>>> > import java.io.UnsupportedEncodingException;
>> >> >>>> > import java.util.List;
>> >> >>>> >
>> >> >>>> > import org.apache.cassandra.thrift.Cassandra;
>> >> >>>> > import org.apache.cassandra.thrift.Column;
>> >> >>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>> >> >>>> > import org.apache.cassandra.thrift.ColumnParent;
>> >> >>>> > import org.apache.cassandra.thrift.ColumnPath;
>> >> >>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>> >> >>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>> >> >>>> > import org.apache.cassandra.thrift.NotFoundException;
>> >> >>>> > import org.apache.cassandra.thrift.SlicePredicate;
>> >> >>>> > import org.apache.cassandra.thrift.SliceRange;
>> >> >>>> > import org.apache.cassandra.thrift.SuperColumn;
>> >> >>>> > import org.apache.cassandra.thrift.TimedOutException;
>> >> >>>> > import org.apache.cassandra.thrift.UnavailableException;
>> >> >>>> > import org.apache.thrift.TException; import
>> >> >>>> > org.apache.thrift.protocol.TBinaryProtocol;
>> >> >>>> > import org.apache.thrift.protocol.TProtocol;
>> >> >>>> > import org.apache.thrift.transport.TSocket;
>> >> >>>> > import org.apache.thrift.transport.TTransport;
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > public class CrudTest {
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private static final String KEYSPACE = "Keyspace1";
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        public static void main(String[] args) {
>> >> >>>> >                CrudTest client = new CrudTest();
>> >> >>>> >
>> >> >>>> >                try {
>> >> >>>> >                        client.run();
>> >> >>>> >                } catch (Exception e) {
>> >> >>>> >                        e.printStackTrace();
>> >> >>>> >                }
>> >> >>>> >
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        public void run() throws TException,
>> >> >>>> > InvalidRequestException, UnavailableException,
>> >> >>>> > UnsupportedEncodingException, NotFoundException,
>> >> >>>> > TimedOutException
>> >> >>>> > {
>> >> >>>> >                TTransport tr = new TSocket("localhost", 9160);
>> >> >>>> >                TProtocol proto = new TBinaryProtocol(tr);
>> >> >>>> >                Cassandra.Client client = new
>> >> >>>> > Cassandra.Client(proto);
>> >> >>>> >                tr.open();
>> >> >>>> >
>> >> >>>> >                System.out.println("******** CREATING DATA
>> >> >>>> > *********");
>> >> >>>> >                createData(client);
>> >> >>>> >                getData(client);
>> >> >>>> >                System.out.println();
>> >> >>>> >                System.out.println("******** DELETING DATA
>> >> >>>> > *********");
>> >> >>>> >                deleteData(client);
>> >> >>>> >                getData(client);
>> >> >>>> >                System.out.println();
>> >> >>>> >                System.out.println("******** CREATING DATA
>> >> >>>> > *********");
>> >> >>>> >                createData(client);
>> >> >>>> >                getData(client);
>> >> >>>> >
>> >> >>>> >                tr.close();
>> >> >>>> >          }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private void createData(Cassandra.Client client) throws
>> >> >>>> > InvalidRequestException, UnavailableException,
>> >> >>>> > TimedOutException,
>> >> >>>> > TException {
>> >> >>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>> >> >>>> >                cp1.setSuper_column("hotel".getBytes());
>> >> >>>> >                cp1.setColumn("Best Western".getBytes());
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >                client.insert(KEYSPACE,
>> >> >>>> >                          "name",
>> >> >>>> >                          cp1,
>> >> >>>> >                          "Best Western of SF".getBytes(),
>> >> >>>> >                          System.currentTimeMillis(),
>> >> >>>> >                          ConsistencyLevel.ALL);
>> >> >>>> >
>> >> >>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >> >>>> >                cp2.setSuper_column("hotel".getBytes());
>> >> >>>> >                cp2.setColumn("Econolodge".getBytes());
>> >> >>>> >
>> >> >>>> >                client.insert(KEYSPACE,
>> >> >>>> >                                  "name",
>> >> >>>> >                                  cp2,
>> >> >>>> >                                  "Econolodge of SF".getBytes(),
>> >> >>>> >                                  System.currentTimeMillis(),
>> >> >>>> >                                  ConsistencyLevel.ALL);
>> >> >>>> >
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private void deleteData(Cassandra.Client client) throws
>> >> >>>> > InvalidRequestException, UnavailableException,
>> >> >>>> > TimedOutException,
>> >> >>>> > TException {
>> >> >>>> >
>> >> >>>> >                client.remove(KEYSPACE,
>> >> >>>> >                                  "hotel",
>> >> >>>> >                                  new ColumnPath("Super2"),
>> >> >>>> >                                  System.currentTimeMillis(),
>> >> >>>> >                                  ConsistencyLevel.ONE);
>> >> >>>> >
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private void getData(Cassandra.Client client) throws
>> >> >>>> > InvalidRequestException, UnavailableException,
>> >> >>>> > TimedOutException,
>> >> >>>> > TException {
>> >> >>>> >                SliceRange sliceRange = new SliceRange();
>> >> >>>> >                sliceRange.setStart(new byte[] {});
>> >> >>>> >                sliceRange.setFinish(new byte[] {});
>> >> >>>> >
>> >> >>>> >                SlicePredicate slicePredicate = new
>> >> >>>> > SlicePredicate();
>> >> >>>> >                slicePredicate.setSlice_range(sliceRange);
>> >> >>>> >
>> >> >>>> >                getData(client, slicePredicate);
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private void getData(Cassandra.Client client,
>> >> >>>> > SlicePredicate
>> >> >>>> > slicePredicate) throws InvalidRequestException,
>> >> >>>> > UnavailableException, TimedOutException, TException {
>> >> >>>> >                List<ColumnOrSuperColumn> coscList =
>> >> >>>> > client.get_slice(KEYSPACE, "hotel", new ColumnParent("Super2"),
>> >> >>>> > slicePredicate, ConsistencyLevel.ALL);
>> >> >>>> >
>> >> >>>> >                if (coscList.isEmpty()) {
>> >> >>>> >                        System.out.println("Column Or Super
>> >> >>>> > Column
>> >> >>>> > is EMPTY");
>> >> >>>> >                }
>> >> >>>> >
>> >> >>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>> >> >>>> >
>> >> >>>> >                        if (cosc == null) {
>> >> >>>> >                                System.out.println("NULL RETURN
>> >> >>>> > VALUE");
>> >> >>>> >                        }
>> >> >>>> >
>> >> >>>> >                        SuperColumn superColumn =
>> >> >>>> > cosc.getSuper_column();
>> >> >>>> >
>> >> >>>> >                        if (superColumn == null) {
>> >> >>>> >                                System.out.println("Super Column
>> >> >>>> > is
>> >> >>>> > NULL");
>> >> >>>> >                        }
>> >> >>>> >                        else {
>> >> >>>> >                                showSuperColumnInfo(superColumn);
>> >> >>>> >                        }
>> >> >>>> >
>> >> >>>> >                }
>> >> >>>> >
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private void showSuperColumnInfo(SuperColumn superColumn)
>> >> >>>> > {
>> >> >>>> >                System.out.println("######## Super Columns
>> >> >>>> > ###########");
>> >> >>>> >                System.out.println("Super Column Name = " + new
>> >> >>>> > String(superColumn.getName()));
>> >> >>>> >                List<Column> columnList =
>> >> >>>> > superColumn.getColumns();
>> >> >>>> >
>> >> >>>> >                System.out.println("--------- Start Columns
>> >> >>>> > -----------");
>> >> >>>> >                for (Column column: columnList) {
>> >> >>>> >                        System.out.println("Column Name = " + new
>> >> >>>> > String(column.getName()));
>> >> >>>> >                        System.out.println("Column Value = " +
>> >> >>>> > new
>> >> >>>> > String(column.getValue()));
>> >> >>>> >                }
>> >> >>>> >                System.out.println("--------- End Columns
>> >> >>>> > -----------");
>> >> >>>> >
>> >> >>>> > System.out.println("##################################");
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > -----Original Message-----
>> >> >>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>> >> >>>> > Sent: Thu 7/22/2010 1:56 PM
>> >> >>>> > To: [hidden email]
>> >> >>>> > Subject: Re: CRUD test
>> >> >>>> >
>> >> >>>> > Yes, and that was the issue. But now after I delete a row from
>> >> >>>> > cassandra-cli, I cannot insert anything back with my code.
>> >> >>>> > Insert
>> >> >>>> > code works does not throw any exceptions but when I read just
>> >> >>>> > inserted columns I get NotFoundException at the last line:
>> >> >>>> >
>> >> >>>> >            client = borrowClient();
>> >> >>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>> >> >>>> > CONSISTENCY_LEVEL);
>> >> >>>> >            ColumnPath cp = new ColumnPath(application);
>> >> >>>> >            cp.setSuper_column(uuid.getBytes());
>> >> >>>> >            SuperColumn sc = keyspace.getSuperColumn(category,
>> >> >>>> > cp);
>> >> >>>> >
>> >> >>>> > It makes me think that once I remove supercolumn it cannot be
>> >> >>>> > created again.
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs" <[hidden email]>
>> >> >>>> > wrote:
>> >> >>>> >
>> >> >>>> > Have you checked the timestamp you're using for the subsequent
>> >> >>>> > inserts is higher than that used in the delete?
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>> >> >>>> > <[hidden email]>
>> >> >>>> > wrote:
>> >> >>>> >> Hi there,
>> >> >>>> >> I'm try...
>> >> >>>> > --
>> >> >>>> > Maybe she awoke to see the roommate's boyfriend swinging from
>> >> >>>> > the
>> >> >>>> > chandelier wearing a boar's head.
>> >> >>>> >
>> >> >>>> > Something which you, I, and everyone else would call "Tuesday",
>> >> >>>> > of
>> >> >>>> > course.
>> >> >>>> >
>> >> >>>> >
>> >> >>>
>> >> >>>
>> >> >>
>> >> >
>> >> >
>> >
>> >
>
>
Reply | Threaded
Open this post in threaded view
|

RE: CRUD test

Peter Minearo
One last question for me:

If I wanted to find all the "Keys" within a SuperColumn Family would I do something like:


client.get_slice(KEYSPACE, null, new ColumnParent("Super2"), slicePredicate,  ConsistencyLevel.ALL);

--OR --

client.get_slice(KEYSPACE, "", new ColumnParent("Super2"), slicePredicate,  ConsistencyLevel.ALL);

Or is there a different way to find everything under a SuperColumn Family?



-----Original Message-----
From: Jonathan Shook [mailto:[hidden email]]
Sent: Saturday, July 24, 2010 11:27 PM
To: [hidden email]
Subject: Re: CRUD test

That's a question that many Java developers would like the answer to.
Unfortunately, anything better than milliseconds requires JNI, since the current JVM doesn't officially support anything higher. There are solutions to this particular problem, but for most people, milliseconds are sufficient outside of testing. This is because the likelihood of making two conflicting changes to the same row/column in the same "session" within the same millisecond is pretty low for actual users and real scenarios. Tests tend to be a little unrealistic in the sense that they happen quickly within a short amount of time and aren't dependent on the timing of people or other systems.

If you are using a "remove and replace" scheme, it could still be a problem. The way I get around it for now is to use the microsecond unit of time with a millisecond source (getCurrentMillis()), and increment it artificially when it would return the same value twice in a row. It's a hack, but it works for my purposes.

On Sun, Jul 25, 2010 at 12:54 AM, Oleg Tsvinev <[hidden email]> wrote:

> Thank you guys for your help!
> Yes, I am using System.currentTimeMillis() in my CRUD test. Even
> though I'm still using it my tests now run as expected. I do not use
> cassandra-cli anymore.
> @Ran great job on Hector, I wish there was more documentation but I managed.
> @Jonathan, what is the recommended time source? I use batch_mutation
> to insert and update multiple columns atomically. Do I have to use the 
> batch_mutation for deletion, too?
> On Sat, Jul 24, 2010 at 2:36 PM, Jonathan Shook <[hidden email]> wrote:
>>
>> Just to clarify, microseconds may be used, but they provide the same
>> behavior as milliseconds if they aren't using a higher time
>> resolution underneath. In some cases, the microseconds are generated
>> simply as milliseconds * 1000, which doesn't actually fix any sequencing bugs.
>>
>> On Sat, Jul 24, 2010 at 3:46 PM, Ran Tavory <[hidden email]> wrote:
>> > Hi Oleg, I didn't follow up the entire thread, but just to let you
>> > know that the 0.6.* version of the CLI uses microsec as the time
>> > unit for timestamps.
>> > Hector also uses micros to match that, however, previous versions
>> > of hector (as well as the CLI) used milliseconds, not micro.
>> > So if you're using hector version 0.6.0-11 or earlier, or by any
>> > chance in some other ways are mixing milisec in your app (are you
>> > using
>> > System.currentTimeMili() somewhere?) then the behavior you're
>> > seeing is expected.
>> >
>> > On Sat, Jul 24, 2010 at 1:06 AM, Jonathan Shook <[hidden email]>
>> > wrote:
>> >>
>> >> I think you are getting it.
>> >>
>> >> As far as what means what at which level, it's really about using
>> >> them consistently in every case. The [row] key (or [row] key
>> >> range) is a top-level argument for all of the operations, since it
>> >> is the "key" to mapping the set of responsible nodes. The key is
>> >> the part of the name of any column which most affects how the load
>> >> is apportioned in the cluster, so it is used very early in request processing.
>> >>
>> >>
>> >> On Fri, Jul 23, 2010 at 4:22 PM, Peter Minearo
>> >> <[hidden email]> wrote:
>> >> > Consequentially the remove should look like:
>> >> >
>> >> >
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> >                cp1.setSuper_column("Best Western".getBytes());
>> >> >
>> >> >                client.remove(KEYSPACE,
>> >> >                                  "hotel",
>> >> >                                  cp1,
>> >> >                                  System.currentTimeMillis(),
>> >> >                                  ConsistencyLevel.ONE);
>> >> >
>> >> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >> >                cp2.setSuper_column("Econolodge".getBytes());
>> >> >
>> >> >                client.remove(KEYSPACE,
>> >> >                                  "hotel",
>> >> >                                  cp2,
>> >> >                                  System.currentTimeMillis(),
>> >> >                                  ConsistencyLevel.ONE);
>> >> >
>> >> >
>> >> > -----Original Message-----
>> >> > From: Peter Minearo [mailto:[hidden email]]
>> >> > Sent: Fri 7/23/2010 2:17 PM
>> >> > To: [hidden email]
>> >> > Subject: RE: CRUD test
>> >> >
>> >> > CORRECTION:
>> >> >
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> > cp1.setSuper_column("Best Western".getBytes());
>> >> > cp1.setColumn("name".getBytes()); client.insert(KEYSPACE,
>> >> > "hotel", cp1, "Best Western of SF".getBytes(),
>> >> > System.currentTimeMillis(), ConsistencyLevel.ALL);
>> >> >
>> >> >
>> >> > -----Original Message-----
>> >> > From: Peter Minearo [mailto:[hidden email]]
>> >> > Sent: Friday, July 23, 2010 2:14 PM
>> >> > To: [hidden email]
>> >> > Subject: RE: CRUD test
>> >> >
>> >> > Interesting!! Let me rephrase to make sure I understood what is
>> >> > going
>> >> > on:
>> >> >
>> >> > When Inserting data via the "insert" function/method:
>> >> >
>> >> > void insert(string keyspace, string key, ColumnPath column_path,
>> >> > binary value, i64 timestamp, ConsistencyLevel consistency_level)
>> >> >
>> >> > The "key" parameter is the actual Key to the "Row", which
>> >> > contains SuperColumns.  The 'ColumnPath' gives the path within the Key.
>> >> >
>> >> >
>> >> >
>> >> > INCORRECT:
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> > cp1.setSuper_column("hotel".getBytes());
>> >> > cp1.setColumn("Best Western".getBytes());
>> >> > client.insert(KEYSPACE, "name", cp1, "Best Western of
>> >> > SF".getBytes(), System.currentTimeMillis(),
>> >> > ConsistencyLevel.ALL);
>> >> >
>> >> >
>> >> > CORRECT:
>> >> > ColumnPath cp1 = new ColumnPath("Super2");
>> >> > cp1.setSuper_column("name".getBytes());
>> >> > cp1.setColumn("Best Western".getBytes());
>> >> > client.insert(KEYSPACE, "hotel", cp1, "Best Western of
>> >> > SF".getBytes(), System.currentTimeMillis(),
>> >> > ConsistencyLevel.ALL);
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > -----Original Message-----
>> >> > From: Jonathan Shook [mailto:[hidden email]]
>> >> > Sent: Friday, July 23, 2010 1:49 PM
>> >> > To: [hidden email]
>> >> > Subject: Re: CRUD test
>> >> >
>> >> > Correct.
>> >> >
>> >> > After the initial insert,
>> >> >
>> >> > cassandra> get Keyspace1.Super2['name']
>> >> > => (super_column=hotel,
>> >> >     (column=Best Western, value=Best Western of SF,
>> >> > timestamp=1279916772571)
>> >> >     (column=Econolodge, value=Econolodge of SF,
>> >> > timestamp=1279916772573)) Returned 1 results.
>> >> >
>> >> > ... and ...
>> >> >
>> >> > cassandra> get Keyspace1.Super2['hotel']
>> >> > Returned 0 results.
>> >> >
>> >> >
>> >> >
>> >> > On Fri, Jul 23, 2010 at 3:41 PM, Peter Minearo
>> >> > <[hidden email]> wrote:
>> >> >> The Model Should look like:
>> >> >>
>> >> >>
>> >> >> Super2 = {
>> >> >>        hotel: {
>> >> >>                        Best Western: {name: "Best Western of
>> >> >> SF"}
>> >> >>                        Econolodge: {name: "Econolodge of SF"}
>> >> >>        }
>> >> >> }
>> >> >>
>> >> >> Are the CRUD Operations not referencing this correctly?
>> >> >>
>> >> >>
>> >> >>
>> >> >> -----Original Message-----
>> >> >> From: Jonathan Shook [mailto:[hidden email]]
>> >> >> Sent: Friday, July 23, 2010 1:34 PM
>> >> >> To: [hidden email]
>> >> >> Subject: Re: CRUD test
>> >> >>
>> >> >> There seem to be data consistency bugs in the test.  Are "name"
>> >> >> and "hotel" being used in a pair-wise way?
>> >> >> Specifically, the first test is using creating one and checking
>> >> >> for the other.
>> >> >>
>> >> >> On Fri, Jul 23, 2010 at 2:46 PM, Oleg Tsvinev
>> >> >> <[hidden email]>
>> >> >> wrote:
>> >> >>> Johathan,
>> >> >>> I followed your suggestion. Unfortunately, CRUD test still
>> >> >>> does not work for me. Can you provide a simplest CRUD test
>> >> >>> possible that works?
>> >> >>> On Fri, Jul 23, 2010 at 10:59 AM, Jonathan Shook
>> >> >>> <[hidden email]>
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> I suspect that it is still your timestamps.
>> >> >>>> You can verify this with a fake timestamp generator that is
>> >> >>>> simply incremented on each getTimestamp().
>> >> >>>>
>> >> >>>> 1 millisecond is a long time for code that is wrapped tightly
>> >> >>>> in a test. You are likely using the same logical time stamp
>> >> >>>> for multiple operations.
>> >> >>>>
>> >> >>>>
>> >> >>>> On Thu, Jul 22, 2010 at 6:29 PM, Peter Minearo
>> >> >>>> <[hidden email]> wrote:
>> >> >>>> > I am able to reproduce his problem. If you take the default
>> >> >>>> > storage-conf.xml file and utilize the "Super2" ColumnFamily
>> >> >>>> > with the code below.  You will see that the data is not
>> >> >>>> > getting created once you run the delete.  It seems to not
>> >> >>>> > allow you to create data via Thrift.  HOWEVER, data can be
>> >> >>>> > created via the command line tool.
>> >> >>>> >
>> >> >>>> > import java.io.UnsupportedEncodingException;
>> >> >>>> > import java.util.List;
>> >> >>>> >
>> >> >>>> > import org.apache.cassandra.thrift.Cassandra;
>> >> >>>> > import org.apache.cassandra.thrift.Column;
>> >> >>>> > import org.apache.cassandra.thrift.ColumnOrSuperColumn;
>> >> >>>> > import org.apache.cassandra.thrift.ColumnParent;
>> >> >>>> > import org.apache.cassandra.thrift.ColumnPath;
>> >> >>>> > import org.apache.cassandra.thrift.ConsistencyLevel;
>> >> >>>> > import org.apache.cassandra.thrift.InvalidRequestException;
>> >> >>>> > import org.apache.cassandra.thrift.NotFoundException;
>> >> >>>> > import org.apache.cassandra.thrift.SlicePredicate;
>> >> >>>> > import org.apache.cassandra.thrift.SliceRange;
>> >> >>>> > import org.apache.cassandra.thrift.SuperColumn;
>> >> >>>> > import org.apache.cassandra.thrift.TimedOutException;
>> >> >>>> > import org.apache.cassandra.thrift.UnavailableException;
>> >> >>>> > import org.apache.thrift.TException; import
>> >> >>>> > org.apache.thrift.protocol.TBinaryProtocol;
>> >> >>>> > import org.apache.thrift.protocol.TProtocol;
>> >> >>>> > import org.apache.thrift.transport.TSocket;
>> >> >>>> > import org.apache.thrift.transport.TTransport;
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > public class CrudTest {
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private static final String KEYSPACE = "Keyspace1";
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        public static void main(String[] args) {
>> >> >>>> >                CrudTest client = new CrudTest();
>> >> >>>> >
>> >> >>>> >                try {
>> >> >>>> >                        client.run();
>> >> >>>> >                } catch (Exception e) {
>> >> >>>> >                        e.printStackTrace();
>> >> >>>> >                }
>> >> >>>> >
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        public void run() throws TException,
>> >> >>>> > InvalidRequestException, UnavailableException,
>> >> >>>> > UnsupportedEncodingException, NotFoundException,
>> >> >>>> > TimedOutException {
>> >> >>>> >                TTransport tr = new TSocket("localhost",
>> >> >>>> > 9160);
>> >> >>>> >                TProtocol proto = new TBinaryProtocol(tr);
>> >> >>>> >                Cassandra.Client client = new
>> >> >>>> > Cassandra.Client(proto);
>> >> >>>> >                tr.open();
>> >> >>>> >
>> >> >>>> >                System.out.println("******** CREATING DATA
>> >> >>>> > *********");
>> >> >>>> >                createData(client);
>> >> >>>> >                getData(client);
>> >> >>>> >                System.out.println();
>> >> >>>> >                System.out.println("******** DELETING DATA
>> >> >>>> > *********");
>> >> >>>> >                deleteData(client);
>> >> >>>> >                getData(client);
>> >> >>>> >                System.out.println();
>> >> >>>> >                System.out.println("******** CREATING DATA
>> >> >>>> > *********");
>> >> >>>> >                createData(client);
>> >> >>>> >                getData(client);
>> >> >>>> >
>> >> >>>> >                tr.close();
>> >> >>>> >          }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private void createData(Cassandra.Client client)
>> >> >>>> > throws InvalidRequestException, UnavailableException,
>> >> >>>> > TimedOutException, TException {
>> >> >>>> >                ColumnPath cp1 = new ColumnPath("Super2");
>> >> >>>> >                cp1.setSuper_column("hotel".getBytes());
>> >> >>>> >                cp1.setColumn("Best Western".getBytes());
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >                client.insert(KEYSPACE,
>> >> >>>> >                          "name",
>> >> >>>> >                          cp1,
>> >> >>>> >                          "Best Western of SF".getBytes(),
>> >> >>>> >                          System.currentTimeMillis(),
>> >> >>>> >                          ConsistencyLevel.ALL);
>> >> >>>> >
>> >> >>>> >                ColumnPath cp2 = new ColumnPath("Super2");
>> >> >>>> >                cp2.setSuper_column("hotel".getBytes());
>> >> >>>> >                cp2.setColumn("Econolodge".getBytes());
>> >> >>>> >
>> >> >>>> >                client.insert(KEYSPACE,
>> >> >>>> >                                  "name",
>> >> >>>> >                                  cp2,
>> >> >>>> >                                  "Econolodge of
>> >> >>>> > SF".getBytes(),
>> >> >>>> >                                  
>> >> >>>> > System.currentTimeMillis(),
>> >> >>>> >                                  ConsistencyLevel.ALL);
>> >> >>>> >
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private void deleteData(Cassandra.Client client)
>> >> >>>> > throws InvalidRequestException, UnavailableException,
>> >> >>>> > TimedOutException, TException {
>> >> >>>> >
>> >> >>>> >                client.remove(KEYSPACE,
>> >> >>>> >                                  "hotel",
>> >> >>>> >                                  new ColumnPath("Super2"),
>> >> >>>> >                                  
>> >> >>>> > System.currentTimeMillis(),
>> >> >>>> >                                  ConsistencyLevel.ONE);
>> >> >>>> >
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private void getData(Cassandra.Client client) throws
>> >> >>>> > InvalidRequestException, UnavailableException,
>> >> >>>> > TimedOutException, TException {
>> >> >>>> >                SliceRange sliceRange = new SliceRange();
>> >> >>>> >                sliceRange.setStart(new byte[] {});
>> >> >>>> >                sliceRange.setFinish(new byte[] {});
>> >> >>>> >
>> >> >>>> >                SlicePredicate slicePredicate = new
>> >> >>>> > SlicePredicate();
>> >> >>>> >                slicePredicate.setSlice_range(sliceRange);
>> >> >>>> >
>> >> >>>> >                getData(client, slicePredicate);
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private void getData(Cassandra.Client client,
>> >> >>>> > SlicePredicate
>> >> >>>> > slicePredicate) throws InvalidRequestException,
>> >> >>>> > UnavailableException, TimedOutException, TException {
>> >> >>>> >                List<ColumnOrSuperColumn> coscList =
>> >> >>>> > client.get_slice(KEYSPACE, "hotel", new
>> >> >>>> > ColumnParent("Super2"), slicePredicate,
>> >> >>>> > ConsistencyLevel.ALL);
>> >> >>>> >
>> >> >>>> >                if (coscList.isEmpty()) {
>> >> >>>> >                        System.out.println("Column Or Super
>> >> >>>> > Column is EMPTY");
>> >> >>>> >                }
>> >> >>>> >
>> >> >>>> >                for (ColumnOrSuperColumn cosc: coscList) {
>> >> >>>> >
>> >> >>>> >                        if (cosc == null) {
>> >> >>>> >                                System.out.println("NULL
>> >> >>>> > RETURN VALUE");
>> >> >>>> >                        }
>> >> >>>> >
>> >> >>>> >                        SuperColumn superColumn =
>> >> >>>> > cosc.getSuper_column();
>> >> >>>> >
>> >> >>>> >                        if (superColumn == null) {
>> >> >>>> >                                System.out.println("Super
>> >> >>>> > Column is NULL");
>> >> >>>> >                        }
>> >> >>>> >                        else {
>> >> >>>> >                                
>> >> >>>> > showSuperColumnInfo(superColumn);
>> >> >>>> >                        }
>> >> >>>> >
>> >> >>>> >                }
>> >> >>>> >
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >        private void showSuperColumnInfo(SuperColumn
>> >> >>>> > superColumn) {
>> >> >>>> >                System.out.println("######## Super Columns
>> >> >>>> > ###########");
>> >> >>>> >                System.out.println("Super Column Name = " +
>> >> >>>> > new String(superColumn.getName()));
>> >> >>>> >                List<Column> columnList =
>> >> >>>> > superColumn.getColumns();
>> >> >>>> >
>> >> >>>> >                System.out.println("--------- Start Columns
>> >> >>>> > -----------");
>> >> >>>> >                for (Column column: columnList) {
>> >> >>>> >                        System.out.println("Column Name = "
>> >> >>>> > + new String(column.getName()));
>> >> >>>> >                        System.out.println("Column Value = "
>> >> >>>> > + new String(column.getValue()));
>> >> >>>> >                }
>> >> >>>> >                System.out.println("--------- End Columns
>> >> >>>> > -----------");
>> >> >>>> >
>> >> >>>> > System.out.println("##################################");
>> >> >>>> >        }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > }
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > -----Original Message-----
>> >> >>>> > From: Oleg Tsvinev [mailto:[hidden email]]
>> >> >>>> > Sent: Thu 7/22/2010 1:56 PM
>> >> >>>> > To: [hidden email]
>> >> >>>> > Subject: Re: CRUD test
>> >> >>>> >
>> >> >>>> > Yes, and that was the issue. But now after I delete a row
>> >> >>>> > from cassandra-cli, I cannot insert anything back with my code.
>> >> >>>> > Insert
>> >> >>>> > code works does not throw any exceptions but when I read
>> >> >>>> > just inserted columns I get NotFoundException at the last line:
>> >> >>>> >
>> >> >>>> >            client = borrowClient();
>> >> >>>> >            Keyspace keyspace = client.getKeyspace(KEYSPACE,
>> >> >>>> > CONSISTENCY_LEVEL);
>> >> >>>> >            ColumnPath cp = new ColumnPath(application);
>> >> >>>> >            cp.setSuper_column(uuid.getBytes());
>> >> >>>> >            SuperColumn sc =
>> >> >>>> > keyspace.getSuperColumn(category, cp);
>> >> >>>> >
>> >> >>>> > It makes me think that once I remove supercolumn it cannot
>> >> >>>> > be created again.
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > On Jul 22, 2010 1:13 AM, "Colin Vipurs"
>> >> >>>> > <[hidden email]>
>> >> >>>> > wrote:
>> >> >>>> >
>> >> >>>> > Have you checked the timestamp you're using for the
>> >> >>>> > subsequent inserts is higher than that used in the delete?
>> >> >>>> >
>> >> >>>> >
>> >> >>>> > On Thu, Jul 22, 2010 at 2:29 AM, Oleg Tsvinev
>> >> >>>> > <[hidden email]>
>> >> >>>> > wrote:
>> >> >>>> >> Hi there,
>> >> >>>> >> I'm try...
>> >> >>>> > --
>> >> >>>> > Maybe she awoke to see the roommate's boyfriend swinging
>> >> >>>> > from the chandelier wearing a boar's head.
>> >> >>>> >
>> >> >>>> > Something which you, I, and everyone else would call
>> >> >>>> > "Tuesday", of course.
>> >> >>>> >
>> >> >>>> >
>> >> >>>
>> >> >>>
>> >> >>
>> >> >
>> >> >
>> >
>> >
>
>