*************************************************
Data Cafe Deployment and Evaluation in Amazon EC2
*************************************************

Data Cafe is deployed in Amazon EC2 along with the data sources and tools such as Drill for its evaluation.


Current EC2 Deployment
######################

1 node for Mongo.

1 node for Java - datacafe.

1 node for Drill + EMR Hadoop/Hive master.

2 nodes for EMR Hadoop/Hive core. (Can be more than 2, with no code change. Currently 0 Task instances.

Task instances can be introduced without code changes as well).

Currently, 5 EC2 nodes in total.


Configure Data Sources in AWS
#############################

**Configure Mongo in EC2.**

* Connect to the remote EC2/Mongo Instance

 $ ssh -i "pradeeban.pem" ubuntu@ec2-54-166-90-207.compute-1.amazonaws.com


* Copy files to the remote instance.

 $ scp -i "pradeeban.pem" /home/pradeeban/gsoc2015/conf/clinical.csv ubuntu@ec2-54-157-62-174.compute-1.amazonaws.com:/home/ubuntu

 $ scp -i "pradeeban.pem" /home/pradeeban/gsoc2015/conf/pathology.csv ubuntu@ec2-54-157-62-174.compute-1.amazonaws.com:/home/ubuntu


**Execute in Amazon EC2 Instances and Elastic MapReduce (Amazon EMR).**

* SSH to remote instance

 $ ssh -i "pradeeban.pem" ubuntu@ec2-54-157-222-55.compute-1.amazonaws.com:/home/ubuntu/


* Copy All to the EC2 Instance

 $ scp -r -i "pradeeban.pem" . ubuntu@ec2-54-157-222-55.compute-1.amazonaws.com:/home/ubuntu/datacafe


* Copy only the datacafe jar

 $ scp -i "pradeeban.pem" lib/datacafe-server-1.0-SNAPSHOT.jar ubuntu@ec2-54-157-222-55.compute-1.amazonaws.com:/home/ubuntu/datacafe/lib


**Configure Drill in EMR.**

* Launch Drill during EMR instance bootstrap action

 s3://maprtech-emr/scripts/mapr_drill_bootstrap.sh