Required skills:

• Hadoop/Spark – Strong hands-on experience with the Hadoop ecosystem and real-time/batch processing using Spark.

• Kafka – Expertise in building and managing streaming data pipelines.

• Python – Proficient in writing efficient data processing and transformation scripts.

• NoSQL Databases – Experience with MongoDB, HBase, or SingleStore for handling large-scale, unstructured data.

• UNIX / Shell Scripting – Ability to automate tasks and manage systems efficiently in a Linux/Unix environment.

Day-to-Day Responsibilities:

• Design, develop, and implement scalable Big Data solutions using Hadoop, Spark, and Kafka.

• Collaborate with business stakeholders to gather requirements and conceptualize technical solutions.

• Build real-time and batch data pipelines to support large-scale data processing and analytics.

• Optimize data workflows using tools like Hive, Sqoop, Impala, and Oozie.

• Perform performance benchmarking and resolve system bottlenecks.

• Develop Python and Shell scripts for data transformation, automation, and orchestration.

• Integrate NoSQL databases into analytics workflows for diverse data storage needs.

• Participate in architecture discussions and define Big Data strategies aligned with business goals.

• Work with offshore/onshore teams to develop, test, and deploy solutions within delivery timelines.

• Mentor junior team members and stay current on emerging Big Data technologies and trends.
  
  
  PayRate Range: $60/HR-$65/HR