Senior Software Engineer (MERN) | Datics

Software Engineer (Python)

Website Datics AI

Building Great Tech Products

We are looking for a Software Engineer (Python) specializing in Web Scraping to join our team and help us develop and maintain various scraping products which will fuel our Big Data Market Analytics Platform. The Software Engineer (Python) will be responsible for designing, developing, and implementing various scrapers to extract data from simple to complex websites and maintaining Data Pipelines.

Experience: 2.5 years

Education: BSCS or relevant degree in IT

Requirements:
– 2+ years of work experience in large-scale scraping projects
– Good knowledge of Various Databases from RDBMS to NoSQL ( MSSQL, MongoDB)
– Good understanding of Data structures and Algorithms
– Good Experience in Web Scraping framework Scrapy, Beautiful soup, Splash or other frameworks and related libraries
– Deploy the python scripts/crawlers on Linux-based servers
– Ability to inspect and replicate the API calls in the scrapy spiders with required payloads and session variables.
– Knowledge to parse HTML by using complex paths and JSON outputs with JSON modules.
– Capable of quickly learning and implementing new tools/technologies
– Experience with Django is a huge plus

Responsibilities:
– Analyze and design different data models/tables to store crawlers output.
– Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL
– Create data normalization, data enrichment, and QA pipelines by using different modules in python to clean and normalize raw data scraped from different sources.-
– Write complex SQL queries or procedures to manipulate/merge the data
– Deploy the python scripts to Linux-based AWS servers
– Manage the performance of MySQL databases by performance tuning, query optimization & index tuning.

Working at Datics means joining a family of close-knit professionals. All of us work in an environment that fosters growth, respect, inclusivity, and complete autonomy and freedom in the ability to exercise one’s skills and expertise.

In order to apply for this job, email your details to  talent@datics.ai

To apply for this job email your details to hr@datics.ai

Author

Admin