Role:Tech Lead - Data Science

Location:Indore/Remote

Employement Type:Full Time

Department:Development

Salary: 75000 - 150000

Job Description

We are looking for a Tech Lead – Python Developer / Data Engineer / Machine Learning Engineer who can lead a team, manage projects, and build AI-driven scalable applications. The ideal candidate should have expertise in backend development, data engineering, machine learning, and large-scale AI architectures, including Vector Databases and Large Language Models (LLMs).

You will work on designing and implementing high-performance applications, real-time streaming, AI pipelines, and vector-based search solutions while guiding a team of developers.

Skill & Qualification

Required Skills & Qualifications:

• 5+ years of experience in Python backend development, data engineering & AI.
• Strong expertise in Vector Databases (FAISS, Milvus, Weaviate, Pinecone).
• Experience in LLM-based applications, Transformer models, and NLP pipelines.
• Proficiency in MongoDB, PostgreSQL, Redis, and Elasticsearch.
• Strong experience with data processing, ETL workflows, and streaming data.
• Hands-on experience with Celery, Kafka, RabbitMQ, Redis Pub/Sub, or similar message brokers.
• Experience in Ubuntu/Linux server management.
• Hands-on experience in Docker, Kubernetes, and CI/CD pipelines.
• Strong problem-solving, algorithmic thinking, and leadership skills.

Nice to Have (Preferred Skills):

• Experience with Graph databases (Neo4j, ArangoDB).
• Knowledge of Serverless architectures (AWS Lambda, GCP Cloud Functions).
• Familiarity with Big Data tools (Hadoop, Dask, Apache Beam).
• Experience with LangChain and RAG-based AI applications.
• Knowledge of Airflow or Prefect for workflow orchestration.

Roles & Responsibilities

Leadership & Team Management:

• Lead a team of Python developers, Data Engineers, and AI/ML engineers.
• Architect and oversee the development of scalable AI-driven applications.
• Conduct code reviews, enforce best practices, and mentor junior developers.
• Collaborate with stakeholders to define and execute LLM-based solutions.

Backend Development & Data Engineering:

• Architect and develop scalable backend systems using Python.
• Design and optimize REST APIs, WebSockets, and GraphQL services.
• Develop ETL pipelines for real-time and batch data processing.
• Implement task scheduling & distributed computing using Celery, Kafka, or Airflow.
• Work with message queues (Kafka, RabbitMQ, Redis Pub/Sub) for event-driven architectures.
• Optimize system performance using multi-threading, multiprocessing, and async programming.
• Design and manage data warehouses and data lakes (e.g., Snowflake, Delta Lake, BigQuery).

Machine Learning, LLMs & Vector Databases:

• Develop and deploy LLM-based applications using OpenAI, Hugging Face, or custom models.
• Work with RAG (Retrieval-Augmented Generation) pipelines for AI applications.
• Implement vector-based search using FAISS, Milvus, Weaviate, or Pinecone.
• Train and fine-tune Transformer models (BERT, GPT, LLaMA, etc.) for NLP tasks.
• Deploy ML models using MLflow, Kubeflow, TensorFlow Serving, or FastAPI.
• Optimize data pipelines and embeddings for efficient LLM performance.

Streaming & Real-Time Processing:

• Work with Kafka, Apache Flink, Spark Streaming, and Redis Streams for real-time data processing.
• Implement real-time AI applications using WebSockets & asynchronous processing.
• Design and optimize LLM inference pipelines for low-latency applications.

Databases & Data Storage:

• Expertise in MongoDB, Redis, and Elasticsearch.
• Experience with Vector Databases for AI-driven applications.
• Work with NoSQL & SQL databases, ensuring efficient indexing and query optimization.
• Implement sharding, partitioning, and caching strategies for large-scale systems.

Cloud & DevOps:

• Deploy and manage applications on AWS, GCP, or Azure.
• Set up CI/CD pipelines (GitHub Actions, Jenkins, GitLab CI/CD) for automated deployments.
• Containerize applications using Docker and orchestrate them with Kubernetes.
• Implement infrastructure as code (Terraform, Ansible, CloudFormation) for scalable environments.

Security & Performance Optimization:

• Ensure application security using OAuth, JWT, SSL/TLS, and encryption techniques.
• Perform profiling, logging, and monitoring using Prometheus, Grafana, and ELK Stack.
• Optimize memory usage, query performance, and API response times.