VerSe Innovation Looking for Senior Data Engineer at Bangalore Urban Full Time

About Josh-

Josh is a made-in-India, short-video app launched in September 2020 It celebrates the mahagathbandhan of 200+ exclusive top creators, India’s biggest music labels, the hottest entertainment format, and formidable user demographics. Currently, Josh is the fastest growing and most engaged short video app in India with 80 million DAUs and 30 minutes time spent per DAU clocking 50+ million downloads, 4.2+ billion video plays per day and over 15 million Unique User Generated Content (UGC) content pieces. Its users get to enjoy bite-sized videos up to 120 seconds ranging across categories including viral, trending, glamour, dance, devotion, yoga and cooking, among others. The Josh app is available on Android and iOS.

Responsibilities:

– Design and execute the data pipeline with both batch and real-time analytics capabilities to handle over 5 billion daily events and data in the petabyte scale.
– Engineer scalable, secure, and optimized data transformation pipelines while ensuring seamless integration with downstream systems.
– Implement large-scale data quality audits, develop alert mechanisms, and integrate anomaly detection.
– Flexibly adapt to evolving requirements and demonstrate willingness to explore diverse technologies.
– Enhance data quality and ensure the utmost reliability of all data pipelines.
– Promote standards, best practices, and operational excellence for the data platform.
– Mentor and provide guidance to junior Data Engineers.

Requirements:

– Minimum of 3+ years experience
– Relevant experience of 3+ in Data Engineering.
– Proficiency in designing data platforms, covering aspects such as architecture, storage, query processing, and observability.
– Solid experience with Spark or Flink

– Robust Java/Scala programming skills with a demonstrable understanding of OOPs and functional programming principles.
– Familiarity with the Lakehouse architectural paradigm.
– In-depth expertise in constructing and refining data pipelines, architectures, and data sets.
– Hands-on experience with messaging systems like Kafka, Kinesis, or Rabbit MQ.
– Familiarity with orchestration tools such as Airflow, Oozie, or Luigi.
– Exposure with datastores such as Redis, Elasticsearch, or Cassandra.
– Proficiency in creating REST API endpoints for data extraction and consumption.
– Exposure to containerization technologies like Docker or Kubernetes is a distinct advantage.
– Prior experience on prominent Public Cloud platforms is preferable.

▶️ Senior Data Engineer
🖊️ VerSe Innovation
📍 Bangalore Urban