Best way to use a Cassandra Client in a multi-threaded environment?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Best way to use a Cassandra Client in a multi-threaded environment?

JoelPM
Hello,

Are there any recommendations on how to use Cassandra Clients in a multi-threaded front-end application (java)? Is the Client thread-safe or is it best to do a client per thread (or object pool of some sort)?

Thanks,
Joel
Reply | Threaded
Open this post in threaded view
|

Re: Best way to use a Cassandra Client in a multi-threaded environment?

Jonathan Ellis-3
IIRC thrift makes no effort to generate threadsafe code.

which makes sense in an rpc-oriented protocol really.

On Wed, Jul 15, 2009 at 7:25 PM, Joel Meyer<[hidden email]> wrote:
> Hello,
> Are there any recommendations on how to use Cassandra Clients in a
> multi-threaded front-end application (java)? Is the Client thread-safe or is
> it best to do a client per thread (or object pool of some sort)?
> Thanks,
> Joel
Reply | Threaded
Open this post in threaded view
|

Re: Best way to use a Cassandra Client in a multi-threaded environment?

Ian Holsman-3

On 16/07/2009, at 10:35 AM, Jonathan Ellis wrote:

> IIRC thrift makes no effort to generate threadsafe code.
>
> which makes sense in an rpc-oriented protocol really.

hmm.. not really. you can have a webserver calling a thrift backend  
quite easily, and then you would have 100+ threads all calling the  
same code.

>
> On Wed, Jul 15, 2009 at 7:25 PM, Joel Meyer<[hidden email]>  
> wrote:
>> Hello,
>> Are there any recommendations on how to use Cassandra Clients in a
>> multi-threaded front-end application (java)? Is the Client thread-
>> safe or is
>> it best to do a client per thread (or object pool of some sort)?
>> Thanks,
>> Joel

--
Ian Holsman
[hidden email]



Reply | Threaded
Open this post in threaded view
|

Re: Best way to use a Cassandra Client in a multi-threaded environment?

Sandeep Tata
From what I remember, in Java, the thrift client object is threadsafe
-- but if multiple threads use the same client, they will serialize on
it.
In my tests a client-per thread has worked fine for a small number of
threads . Depending on what your threads are doing and how many you
have, you may need pooling.


On Wed, Jul 15, 2009 at 5:41 PM, Ian Holsman<[hidden email]> wrote:

>
> On 16/07/2009, at 10:35 AM, Jonathan Ellis wrote:
>
>> IIRC thrift makes no effort to generate threadsafe code.
>>
>> which makes sense in an rpc-oriented protocol really.
>
> hmm.. not really. you can have a webserver calling a thrift backend quite
> easily, and then you would have 100+ threads all calling the same code.
>>
>> On Wed, Jul 15, 2009 at 7:25 PM, Joel Meyer<[hidden email]> wrote:
>>>
>>> Hello,
>>> Are there any recommendations on how to use Cassandra Clients in a
>>> multi-threaded front-end application (java)? Is the Client thread-safe or
>>> is
>>> it best to do a client per thread (or object pool of some sort)?
>>> Thanks,
>>> Joel
>
> --
> Ian Holsman
> [hidden email]
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Best way to use a Cassandra Client in a multi-threaded environment?

Jonathan Ellis-3
In reply to this post by Ian Holsman-3
What I mean is, if you have

client.rpc1()
<block for rpc1 to finish>

it doesn't really matter if you can do

client.rpc2()
<block for rpc1, then rpc2>

from another thread or not, since it's dumb. :)

On Wed, Jul 15, 2009 at 7:41 PM, Ian Holsman<[hidden email]> wrote:

>
> On 16/07/2009, at 10:35 AM, Jonathan Ellis wrote:
>
>> IIRC thrift makes no effort to generate threadsafe code.
>>
>> which makes sense in an rpc-oriented protocol really.
>
> hmm.. not really. you can have a webserver calling a thrift backend quite
> easily, and then you would have 100+ threads all calling the same code.
>>
>> On Wed, Jul 15, 2009 at 7:25 PM, Joel Meyer<[hidden email]> wrote:
>>>
>>> Hello,
>>> Are there any recommendations on how to use Cassandra Clients in a
>>> multi-threaded front-end application (java)? Is the Client thread-safe or
>>> is
>>> it best to do a client per thread (or object pool of some sort)?
>>> Thanks,
>>> Joel
>
> --
> Ian Holsman
> [hidden email]
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Best way to use a Cassandra Client in a multi-threaded environment?

Ian Holsman-3
ugh.
if this is a byproduct of thrift, we should have another way of  
getting to the backend.
serialization is *not* a desired feature for most people ;-0

On 16/07/2009, at 11:06 AM, Jonathan Ellis wrote:

> What I mean is, if you have
>
> client.rpc1()
> <block for rpc1 to finish>
>
> it doesn't really matter if you can do
>
> client.rpc2()
> <block for rpc1, then rpc2>
>
> from another thread or not, since it's dumb. :)
>
> On Wed, Jul 15, 2009 at 7:41 PM, Ian Holsman<[hidden email]> wrote:
>>
>> On 16/07/2009, at 10:35 AM, Jonathan Ellis wrote:
>>
>>> IIRC thrift makes no effort to generate threadsafe code.
>>>
>>> which makes sense in an rpc-oriented protocol really.
>>
>> hmm.. not really. you can have a webserver calling a thrift backend  
>> quite
>> easily, and then you would have 100+ threads all calling the same  
>> code.
>>>
>>> On Wed, Jul 15, 2009 at 7:25 PM, Joel Meyer<[hidden email]>  
>>> wrote:
>>>>
>>>> Hello,
>>>> Are there any recommendations on how to use Cassandra Clients in a
>>>> multi-threaded front-end application (java)? Is the Client thread-
>>>> safe or
>>>> is
>>>> it best to do a client per thread (or object pool of some sort)?
>>>> Thanks,
>>>> Joel
>>
>> --
>> Ian Holsman
>> [hidden email]
>>
>>
>>
>>

--
Ian Holsman
[hidden email]



Reply | Threaded
Open this post in threaded view
|

Re: Best way to use a Cassandra Client in a multi-threaded environment?

Jonathan Ellis-3
On Wed, Jul 15, 2009 at 8:13 PM, Ian Holsman<[hidden email]> wrote:
> ugh.
> if this is a byproduct of thrift

it is.

> we should have another way of getting to
> the backend.
> serialization is *not* a desired feature for most people ;-0

maybe not, but that's how every single database client works that I
can think of, so it shouldn't exactly be surprising.

you want multiple commands executing in parallel, you open multiple
connections.  not a Big Deal imo.

-Jonathan
Reply | Threaded
Open this post in threaded view
|

Re: Best way to use a Cassandra Client in a multi-threaded environment?

JoelPM
Thanks for the input. Sounds like pooling or thread locals is the way to go.

Cheers,
Joel

On Wed, Jul 15, 2009 at 6:57 PM, Jonathan Ellis <[hidden email]> wrote:
On Wed, Jul 15, 2009 at 8:13 PM, Ian Holsman<[hidden email]> wrote:
> ugh.
> if this is a byproduct of thrift

it is.

> we should have another way of getting to
> the backend.
> serialization is *not* a desired feature for most people ;-0

maybe not, but that's how every single database client works that I
can think of, so it shouldn't exactly be surprising.

you want multiple commands executing in parallel, you open multiple
connections.  not a Big Deal imo.

-Jonathan