Quantcast

Running across multiple EC2 regions...

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Running across multiple EC2 regions...

Chris Marino
Hello, 

Me and some of my colleagues are about to start some experiments running Cassandra across EC2 regions using virtual networks and have some questions about how this is going to work.

I've read these threads about patching the .yaml file to bind to the Listen address to the public IP...


And this EC2 Snitch patch that lets it work across regions...


I'm pretty familiar with EC2 networking, but not very familiar how Cassandra will use the RPC and Listen ports with this Snitch.

So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. 

What I'm trying to do is set up a virtual network that lets all the nodes use private IPs, but can still communicate across regions. 

We're going to try this with a virtual network as well as with the EC2 Snitch to see how things compare. Being able to use the EC2 private interface is going to make big difference.

Thanks in advance.
CM

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Running across multiple EC2 regions...

Vijay-19-3
"So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. "

If you use 2452, communication within a region is via private ip and communication between the regions are public (Handshaking or intial communication will still be via public ip). In EC2 they dont have 2 interface but they nat the public IP.... even then this patch will do the right thing for you.

There is comments in the patch....

+ * 1) Snitch will automatically set the public IP by querying the AWS API
+ * 
+ * 2) Snitch will set the private IP as a Gossip application state.
+ * 
+ * 3) Snitch implements IESCS and will reset the connection if it is within the
+ * same region to communicate via private IP.
+ * 
+ * Implements Ec2Snitch to inherit its functionality and extend it for
+ * Multi-Region.
+ * 
+ * Operational: All the nodes in this cluster needs to be able to (modify the
+ * Security group settings in AWS) communicate via Public IP's.

Regards,
</VJ>



On Fri, Jul 29, 2011 at 2:11 PM, Chris Marino <[hidden email]> wrote:
Hello, 

Me and some of my colleagues are about to start some experiments running Cassandra across EC2 regions using virtual networks and have some questions about how this is going to work.

I've read these threads about patching the .yaml file to bind to the Listen address to the public IP...


And this EC2 Snitch patch that lets it work across regions...


I'm pretty familiar with EC2 networking, but not very familiar how Cassandra will use the RPC and Listen ports with this Snitch.

So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. 

What I'm trying to do is set up a virtual network that lets all the nodes use private IPs, but can still communicate across regions. 

We're going to try this with a virtual network as well as with the EC2 Snitch to see how things compare. Being able to use the EC2 private interface is going to make big difference.

Thanks in advance.
CM


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Running across multiple EC2 regions...

Chris Marino-2
Thanks Vijay, that helps a lot.

FYI,  I did read the comments but didn't understand what I was reading since I don't know what IECSC is?? Googled it but still came up empty.  What is this??

Sorry, if this is obvious, but I'm pretty new to all this....

Thanks
CM

On Fri, Jul 29, 2011 at 3:18 PM, Vijay <[hidden email]> wrote:
"So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. "

If you use 2452, communication within a region is via private ip and communication between the regions are public (Handshaking or intial communication will still be via public ip). In EC2 they dont have 2 interface but they nat the public IP.... even then this patch will do the right thing for you.

There is comments in the patch....
+ * 1) Snitch will automatically set the public IP by querying the AWS API
+ * 
+ * 2) Snitch will set the private IP as a Gossip application state.
+ * 
+ * 3) Snitch implements IESCS and will reset the connection if it is within the
+ * same region to communicate via private IP.
+ * 
+ * Implements Ec2Snitch to inherit its functionality and extend it for
+ * Multi-Region.
+ * 
+ * Operational: All the nodes in this cluster needs to be able to (modify the
+ * Security group settings in AWS) communicate via Public IP's.

Regards,
</VJ>




On Fri, Jul 29, 2011 at 2:11 PM, Chris Marino <[hidden email]> wrote:
Hello, 

Me and some of my colleagues are about to start some experiments running Cassandra across EC2 regions using virtual networks and have some questions about how this is going to work.

I've read these threads about patching the .yaml file to bind to the Listen address to the public IP...


And this EC2 Snitch patch that lets it work across regions...


I'm pretty familiar with EC2 networking, but not very familiar how Cassandra will use the RPC and Listen ports with this Snitch.

So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. 

What I'm trying to do is set up a virtual network that lets all the nodes use private IPs, but can still communicate across regions. 

We're going to try this with a virtual network as well as with the EC2 Snitch to see how things compare. Being able to use the EC2 private interface is going to make big difference.

Thanks in advance.
CM



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Running across multiple EC2 regions...

Vijay-19-3
Chris,

It is internal to cassandra.(IEndpointStateChangeSubscriber)... the snitch basically starts gossiping via public ip and once it sees the Location of the node to be local (it gets notified via IESCS) and then switches to private IP (safely - flush and reconnect). hope this helps... I didnt intend to make it complicated i thought the operations part is what you have to be worried about... (Change the Security group in both the regions to allow the Public IP's)....

Regards,
</VJ>



On Fri, Jul 29, 2011 at 4:37 PM, Chris Marino <[hidden email]> wrote:
Thanks Vijay, that helps a lot.

FYI,  I did read the comments but didn't understand what I was reading since I don't know what IECSC is?? Googled it but still came up empty.  What is this??

Sorry, if this is obvious, but I'm pretty new to all this....

Thanks
CM

On Fri, Jul 29, 2011 at 3:18 PM, Vijay <[hidden email]> wrote:
"So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. "

If you use 2452, communication within a region is via private ip and communication between the regions are public (Handshaking or intial communication will still be via public ip). In EC2 they dont have 2 interface but they nat the public IP.... even then this patch will do the right thing for you.

There is comments in the patch....

+ * 1) Snitch will automatically set the public IP by querying the AWS API
+ * 
+ * 2) Snitch will set the private IP as a Gossip application state.
+ * 
+ * 3) Snitch implements IESCS and will reset the connection if it is within the
+ * same region to communicate via private IP.
+ * 
+ * Implements Ec2Snitch to inherit its functionality and extend it for
+ * Multi-Region.
+ * 
+ * Operational: All the nodes in this cluster needs to be able to (modify the
+ * Security group settings in AWS) communicate via Public IP's.

Regards,
</VJ>




On Fri, Jul 29, 2011 at 2:11 PM, Chris Marino <[hidden email]> wrote:
Hello, 

Me and some of my colleagues are about to start some experiments running Cassandra across EC2 regions using virtual networks and have some questions about how this is going to work.

I've read these threads about patching the .yaml file to bind to the Listen address to the public IP...


And this EC2 Snitch patch that lets it work across regions...


I'm pretty familiar with EC2 networking, but not very familiar how Cassandra will use the RPC and Listen ports with this Snitch.

So, my question is: If we use the EC2 Snitch patch to set up across regions, will the instance's private IP/interface *ever* be used? Or will all traffic always go in and out of the public interface?? Using the public interface is slower and more expensive that the private interface. 

What I'm trying to do is set up a virtual network that lets all the nodes use private IPs, but can still communicate across regions. 

We're going to try this with a virtual network as well as with the EC2 Snitch to see how things compare. Being able to use the EC2 private interface is going to make big difference.

Thanks in advance.
CM




Loading...