Force data to a specific node

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Force data to a specific node

Everton Lima
It is possible to force a data to stay in a specific node?

--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG
Mestrando em Ciência da Computação pela UFG
Programador no LUPA

Reply | Threaded
Open this post in threaded view
|

Re: Force data to a specific node

Edward Sargisson
Why would you want to?


From: Everton Lima <[hidden email]>
To: Cassandra-User <[hidden email]>
Sent: Wed Jan 02 18:03:49 2013
Subject: Force data to a specific node

It is possible to force a data to stay in a specific node?

--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG
Mestrando em Ciência da Computação pela UFG
Programador no LUPA

Reply | Threaded
Open this post in threaded view
|

Re: Force data to a specific node

Everton Lima
We need to do this to minimize the network I/O. We have our own load data balance algorithm. We have some data that is best to process in a local machine.
Is it possible? How?

2013/1/2 Edward Sargisson <[hidden email]>
Why would you want to?


From: Everton Lima <[hidden email]>
To: Cassandra-User <[hidden email]>
Sent: Wed Jan 02 18:03:49 2013
Subject: Force data to a specific node

It is possible to force a data to stay in a specific node?

--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG
Mestrando em Ciência da Computação pela UFG
Programador no LUPA




--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG
Mestrando em Ciência da Computação pela UFG
Programador no LUPA

Reply | Threaded
Open this post in threaded view
|

Re: Force data to a specific node

Aaron Turner
You'd have to use the ordered partitioner or something like that and
choose your row key according to the node you want it placed.

But that's in general a really bad idea because you end up with
unbalanced nodes and hot spots.

That said, are your nodes on a LAN?  I have my 9+3 node cluster (two
datacenters) on 100Mbps ports (which everyone says not to do) and it's
working just fine.  Even node rebuilds haven't been that bad so far.
If you're trying to avoid WAN replication, then use a dedicated
cluster.

On Wed, Jan 2, 2013 at 10:20 AM, Everton Lima <[hidden email]> wrote:

> We need to do this to minimize the network I/O. We have our own load data
> balance algorithm. We have some data that is best to process in a local
> machine.
> Is it possible? How?
>
>
> 2013/1/2 Edward Sargisson <[hidden email]>
>>
>> Why would you want to?
>>
>> ________________________________
>> From: Everton Lima <[hidden email]>
>> To: Cassandra-User <[hidden email]>
>> Sent: Wed Jan 02 18:03:49 2013
>> Subject: Force data to a specific node
>>
>> It is possible to force a data to stay in a specific node?
>>
>> --
>> Everton Lima Aleixo
>> Bacharel em Ciência da Computação pela UFG
>> Mestrando em Ciência da Computação pela UFG
>> Programador no LUPA
>>
>
>
>
> --
> Everton Lima Aleixo
> Bacharel em Ciência da Computação pela UFG
> Mestrando em Ciência da Computação pela UFG
> Programador no LUPA
>



--
Aaron Turner
http://synfin.net/         Twitter: @synfinatic
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Windows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
    -- Benjamin Franklin
"carpe diem quam minimum credula postero"
Reply | Threaded
Open this post in threaded view
|

Re: Force data to a specific node

Edward Capriolo
There is a crazy, very bad, don't do it way to do this. You can set RF=1 and hack the "LocalPartitioner" (because the local partitioner has been made not to do this)

Then the node you connect to and write is the node the data will get stored on.

Its like memcache "do it yourself" style sharding.

Did I say not suggested.

If not "not suggested"


On Wed, Jan 2, 2013 at 2:54 PM, Aaron Turner <[hidden email]> wrote:
You'd have to use the ordered partitioner or something like that and
choose your row key according to the node you want it placed.

But that's in general a really bad idea because you end up with
unbalanced nodes and hot spots.

That said, are your nodes on a LAN?  I have my 9+3 node cluster (two
datacenters) on 100Mbps ports (which everyone says not to do) and it's
working just fine.  Even node rebuilds haven't been that bad so far.
If you're trying to avoid WAN replication, then use a dedicated
cluster.

On Wed, Jan 2, 2013 at 10:20 AM, Everton Lima <[hidden email]> wrote:
> We need to do this to minimize the network I/O. We have our own load data
> balance algorithm. We have some data that is best to process in a local
> machine.
> Is it possible? How?
>
>
> 2013/1/2 Edward Sargisson <[hidden email]>
>>
>> Why would you want to?
>>
>> ________________________________
>> From: Everton Lima <[hidden email]>
>> To: Cassandra-User <[hidden email]>
>> Sent: Wed Jan 02 18:03:49 2013
>> Subject: Force data to a specific node
>>
>> It is possible to force a data to stay in a specific node?
>>
>> --
>> Everton Lima Aleixo
>> Bacharel em Ciência da Computação pela UFG
>> Mestrando em Ciência da Computação pela UFG
>> Programador no LUPA
>>
>
>
>
> --
> Everton Lima Aleixo
> Bacharel em Ciência da Computação pela UFG
> Mestrando em Ciência da Computação pela UFG
> Programador no LUPA
>



--
Aaron Turner
http://synfin.net/         Twitter: @synfinatic
http://tcpreplay.synfin.net/ - Pcap editing and replay tools for Unix & Windows
Those who would give up essential Liberty, to purchase a little temporary
Safety, deserve neither Liberty nor Safety.
    -- Benjamin Franklin
"carpe diem quam minimum credula postero"

Reply | Threaded
Open this post in threaded view
|

RE: Force data to a specific node

DE VITO Dominique
In reply to this post by Everton Lima

Hi Everton,

 

AFAIK, the pb is not forcing data to a specific node, but forcing some kind of data locality.

 

There is things into cql to do it: you define a composite key (K1, K2), and K1 part is used as a rowkey and K2 is used within column name. So, all rows with same K1 are on the same node.

 

See also https://issues.apache.org/jira/browse/CASSANDRA-5054

 

Dominique

 

 

De : Everton Lima [mailto:[hidden email]]
Envoyé : mercredi 2 janvier 2013 19:20
À : [hidden email]
Objet : Re: Force data to a specific node

 

We need to do this to minimize the network I/O. We have our own load data balance algorithm. We have some data that is best to process in a local machine.
Is it possible? How?

2013/1/2 Edward Sargisson <[hidden email]>

Why would you want to?

 


From: Everton Lima <[hidden email]>
To: Cassandra-User <[hidden email]>
Sent: Wed Jan 02 18:03:49 2013
Subject: Force data to a specific node

It is possible to force a data to stay in a specific node?

--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG

Mestrando em Ciência da Computação pela UFG

Programador no LUPA

 




--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG

Mestrando em Ciência da Computação pela UFG

Programador no LUPA

 

Reply | Threaded
Open this post in threaded view
|

Re: Force data to a specific node

Savio Teles
Hi Dominique,

I have the same problem! I would like to place an object in a specific node because I'm working in a spatial application. How should I choose the K1 part to forcing a given object to go to a node?


2013/1/3 DE VITO Dominique <[hidden email]>

Hi Everton,

 

AFAIK, the pb is not forcing data to a specific node, but forcing some kind of data locality.

 

There is things into cql to do it: you define a composite key (K1, K2), and K1 part is used as a rowkey and K2 is used within column name. So, all rows with same K1 are on the same node.

 

See also https://issues.apache.org/jira/browse/CASSANDRA-5054

 

Dominique

 

 

De : Everton Lima [mailto:[hidden email]]
Envoyé : mercredi 2 janvier 2013 19:20
À : [hidden email]
Objet : Re: Force data to a specific node

 

We need to do this to minimize the network I/O. We have our own load data balance algorithm. We have some data that is best to process in a local machine.
Is it possible? How?

2013/1/2 Edward Sargisson <[hidden email]>

Why would you want to?

 


From: Everton Lima <[hidden email]>
To: Cassandra-User <[hidden email]>
Sent: Wed Jan 02 18:03:49 2013
Subject: Force data to a specific node

It is possible to force a data to stay in a specific node?

--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG

Mestrando em Ciência da Computação pela UFG

Programador no LUPA

 




--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG

Mestrando em Ciência da Computação pela UFG

Programador no LUPA

 




--
Atenciosamente,
Sávio S. Teles de Oliveira
voice: +55 62 9136 6996
http://br.linkedin.com/in/savioteles
Mestrando em Ciências da Computação - UFG
Arquiteto de Software
Laboratory for Ubiquitous and Pervasive Applications (LUPA) - UFG
Reply | Threaded
Open this post in threaded view
|

RE: Force data to a specific node

DE VITO Dominique

Hi Sávio,

 

There is no definitive response: it depends on your business model ;-)

I just guess here it should be something like the id of some data root.

 

Take also a look at http://www.datastax.com/dev/blog/schema-in-cassandra-1-1 and look for "partition key", if you want to go through CQL.

 

 

De : Sávio Teles [mailto:[hidden email]]
Envoyé : jeudi 3 janvier 2013 14:58
À : [hidden email]
Objet : Re: Force data to a specific node

 

Hi Dominique,

I have the same problem! I would like to place an object in a specific node because I'm working in a spatial application. How should I choose the K1 part to forcing a given object to go to a node?

2013/1/3 DE VITO Dominique <[hidden email]>

Hi Everton,

 

AFAIK, the pb is not forcing data to a specific node, but forcing some kind of data locality.

 

There is things into cql to do it: you define a composite key (K1, K2), and K1 part is used as a rowkey and K2 is used within column name. So, all rows with same K1 are on the same node.

 

See also https://issues.apache.org/jira/browse/CASSANDRA-5054

 

Dominique

 

 

De : Everton Lima [mailto:[hidden email]]
Envoyé : mercredi 2 janvier 2013 19:20
À : [hidden email]
Objet : Re: Force data to a specific node

 

We need to do this to minimize the network I/O. We have our own load data balance algorithm. We have some data that is best to process in a local machine.
Is it possible? How?

2013/1/2 Edward Sargisson <[hidden email]>

Why would you want to?

 


From: Everton Lima <[hidden email]>
To: Cassandra-User <[hidden email]>
Sent: Wed Jan 02 18:03:49 2013
Subject: Force data to a specific node

It is possible to force a data to stay in a specific node?

--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG

Mestrando em Ciência da Computação pela UFG

Programador no LUPA

 




--
Everton Lima Aleixo
Bacharel em Ciência da Computação pela UFG

Mestrando em Ciência da Computação pela UFG

Programador no LUPA

 




--
Atenciosamente,
Sávio S. Teles de Oliveira

voice: +55 62 9136 6996
http://br.linkedin.com/in/savioteles

Mestrando em Ciências da Computação - UFG
Arquiteto de Software

Laboratory for Ubiquitous and Pervasive Applications (LUPA) - UFG