Senior Software Engineer - Crawler
Full time
ZoomInfo
Responsibilities
- Design and implement scalable, fault-tolerant web crawling and extraction pipeline components.
- Write clean, production-grade Java and Python code.
- Build and operate large-scale ETL/ELT pipelines for data extraction and transformation.
- Work with GCP and AWS cloud infrastructure, primarily on GKE.
- Improve observability, reliability, and operational excellence across contributed systems.
- Partner with product and data science teams to deliver solutions.
- Contribute to code reviews, documentation, and knowledge sharing.
- Keep current with web technologies, anti-crawling mechanisms, and AI-powered extraction approaches.
Requirements
- At least 5 years of professional software engineering experience building production systems.
- Strong knowledge of algorithms, data structures, concurrency, and distributed systems.
- Proficiency in Java and/or Python.
- Experience owning features end-to-end from design through deployment and operation.
- Experience making component-level architectural decisions.
- Hands-on experience with cloud data warehouses such as BigQuery or Snowflake.
- Experience designing and operating large-scale ETL/ELT pipelines.
- Experience with orchestration tools such as Apache Airflow.
- Experience with streaming or event-driven systems such as Apache Kafka.
- Production experience on GCP or AWS, with GCP preferred.
- Hands-on experience with Kubernetes, including GKE or EKS, for distributed workloads.
- Familiarity with infrastructure-as-code tools such as Terraform.
- Strong communication, pragmatic problem-solving, adaptability, and self-direction.
- Web crawling at scale, Scrapy, proxy infrastructure, anti-bot evasion, SERP extraction, AI/LLM-based extraction, or B2B data-company experience is preferred.
Benefits
- Remote work arrangement is indicated by the LI-REMOTE designation.
- Comprehensive benefits are offered, along with holistic mind, body, and lifestyle well-being programs.
Offerta di lavoro pubblicata 10 ore fa
Offerte di lavoro simili
1.250 €/mese
...Senior Golang Software Engineer (Perm, Italy, Remote) This is a permanent full time remote opportunity for those based in Italy. About Nearform Nearform is an independent team of data & AI experts, engineers, and designers who build intelligent digital solutions...SeniorTempo pienoImpiego permanenteRemotoOrario flessibile- What are you working on? The name of the project(s): Game Development, Game Design, iGaming Products At what stage of the project(s)? The game is at the MVP stage Genres: Casino, Casual, Simulation Platforms: Browser An international iGaming development company...SeniorLavoro da remotoTempo pienoStage/TirocinioLavoro ibridoOrario fissoOrario flessibile
- ...back-end systems using modern technologies? We are looking for a Senior Java Developer to join our team! Innovecs is a global digital... ...UK, the EU, Israel, Australia, and Ukraine. Specializing in software solutions, the Innovecs team has experience in Supply Chain, Healthtech...SeniorRemotoOrario flessibile
- ...WellD Italia is looking for a Senior Backend developer to join its dev team in Milano.... ...fast-growing company dedicated to custom software development, strongly focused on state-of... ...passionate and qualified devs and engineers. We value passion, interest in technology...SeniorLavoro da remotoTempo pienoImpiego permanente
- ...with the security team to align the application with organizational security standards. Collaborate directly with the client engineering team as an embedded engineer. Requirements Strong C# and .NET experience, including advanced C# features and nullable reference...SeniorTempo pienoLungo termineRemotoLavoro da casaTurno di notte
- ...We are looking for a Senior SAP ABAP Developer with strong SAP EWM experience in S/4HANA environments. The ideal candidate has solid ABAP OO skills, hands-on experience with RAP , and deep understanding of warehouse and RF processes. Requirements Strong experience...SeniorTempo pieno
113082 $
...Senior Software Engineer, MediaWiki Summary The Wikimedia Foundation is looking for a Senior Software Engineer to join the MediaWiki Platform team. MediaWiki is the open-source platform that powers Wikipedia and its sister projects - impacting hundreds of millions...SeniorTempo pienoImpiego permanenteRemoto- ...cover and their potential bottlenecks Work with deployment engineers to ensure that what you build goes as quickly as possible to the... ...science or related field ~3+ years of experience as a Software Engineer (Backend) ~2+ years of experience using TypeScript...Senior
- ...and working with brilliant people. You are an accomplished senior Python programmer that enjoys challenging projects in mission-critical... ...one of the industry's pioneers in open source with intelligent engineers at every level from engineer to CTO and CEO level. At Canonical...SeniorTempo pienoRemotoOrario flessibile
110.000 €
...About the team: Builder.io's Publish engineering team builds the platform that empowers marketing and content teams to launch pages... ...and AI-driven automation, this is the team for you. As a Senior Software Engineer at Builder.io, you will… Design and build the APIs...SeniorTempo pienoRemoto- ...India) creates quality solutions for its customers by combining software, back office, and knowledge processing operations to fulfill... ...Summary We are seeking a highly motivated and skilled Software Engineer with 5–8 years of professional experience in designing,...SeniorTempo pieno
- ...moment, we cannot offer the same level of stable salaries as large AAA studios. However, our team already includes many experienced Senior artists who are genuinely passionate about creating something greater than just another game. At Trailhead Interactive, we...SeniorLavoro da remotoStage/TirocinioLibero professionista
- ...And do not miss our latest title "Shop & Stuff" to enjoy while playing! Due to the business expansion, we are looking for a Senior VR Unity Developer to join our Team. If you are passionate about gaming, enjoy a pro-active startup vibe, are always on the lookout...SeniorLavoro da remotoStage/Tirocinio
- ...At Cloudbeds, we're not just building software, we’re transforming hospitality. Our intelligently... ...How You'll Make an Impact: As a Senior Software Engineer on the POS team, you'll work directly... ...-first applications, synchronization engines, or conflict-resolution workflows ~...SeniorTempo pienoRemotoTurniWeekend
116633 $
...Summary The Wikimedia Foundation is seeking a Senior Software Engineer to join the team supporting the Wikidata Platform — the structured data backbone of Wikimedia projects and a key part of the global open knowledge ecosystem. You’ll help scale and sustain the...SeniorTempo pienoImpiego permanenteRemoto- ...EVs, heat pumps, and home batteries. The missing piece is software that can connect and orchestrate these devices at scale.... ...energy transition. What we're looking for: We're hiring a Senior Software Engineer to help build the software layer that makes home energy...SeniorTempo pienoStage/TirocinioRemotoLavoro da casaTurniOrario flessibile
- ...EVs, heat pumps, and home batteries. The missing piece is software that can connect and orchestrate these devices at scale.... ...energy transition. What we're looking for: We’re hiring a Senior Backend Engineer to make Enode’s EV platform more reliable, observable, and...SeniorTempo pienoStage/TirocinioRemotoTurniOrario flessibile
- ...subscriptions at scale, combining the agility of a high-growth business with the backing of a global organization. As the Senior Software Engineer, Vendor Integrations, you will be responsible for building and evolving the CloudBlue platform, a microservices-based SaaS...SeniorRemotoOrario flessibile
- ...What are you working on? The name of the project(s): Senior Pixi.js Game Developer At what stage of the project(s)? MVP stage Genres: Casino Platforms: Browser We are an international iGaming development company creating high-quality casino games and complete...SeniorLavoro da remotoTempo pienoStage/TirocinioLavoro ibridoOrario flessibile
- ..., and PCI 4.0—that protects the infrastructure and customer data behind our industry-leading Voice AI API. We're hiring a Senior Software Engineer in the security space to join our IT & Security team as the company's first security engineering role. This role sits at...SeniorTempo pienoStage/TirocinioRemotoOrario flessibile
- ...We are looking for a highly skilled Senior/Lead Unity Developer who is passionate... ...designers, artists, product managers, and other engineers. We are looking for someone who can... ...well-documented C# code following modern software engineering principles. Optimize gameplay...SeniorLavoro da remotoOrario flessibile
- ...office. These include frontend and backend engineers, AI research scientists, and others from... ...work. We are looking for a skilled Software Engineer to join us. What You’ll Do... ...Provider (we use GCP) ~ Experience with web crawlers, large-scale data processing workflows...Lavoro da remotoTurni
- ...Description We’re a remote-first, distributed software company with team members spread across... ...companies build products and grow their engineering teams. Why do people join Clearview?... ...You Are you an exceptionally strong, senior-level developer with an independent...SeniorTempo pienoLibero professionistaRemotoOrario flessibile
92267 $ - 144201 $
...Summary The Wikimedia Foundation is looking for a software engineer to join the Product Safety and Integrity Team. Engineers at the Wikimedia Foundation get to work on technology that truly makes a global impact — powering Wikipedia and its sister projects, which serve...Tempo pienoImpiego permanenteRemoto- ...modernize, and scale the core systems behind DartConnect, the software platform powering competitive darts worldwide. DartConnect... ...scoring pages, and TV displays. How we work We hire pragmatic engineers who can take ownership, work through ambiguity, and ship...SeniorTempo pienoRemoto
- ...from 30 to 60 employees. The Role We're looking for a Senior Fullstack Engineer who thrives on owning products end-to-end. You'll work... ...CDK What We're Looking For ~5+ years of professional software engineering experience, with strong fullstack ownership...SeniorTempo pienoStage/Tirocinio
- ...Design Why apply? Product Engineering at GraphAware offers a unique... ...knowledge graphs, workflow engines, LLMs, and graph-native UIs pushing... ...technical vision alongside a senior engineering team. True... ...will be doing As a Senior Software Engineer, you will shape the...SeniorLavoro da remotoTempo pienoOrario flessibile
80000 $
...work. We create professional-grade AI systems for insurers: software that can read, reason, decide, and act inside real workflows... ...Salary range: $80k – $165k USD* We are looking for smart engineers who love to code, seek challenging problems, and appreciate recognition...SeniorTempo pienoStage/TirocinioRemoto- ...payments to smart transaction management and many more. As a Senior Backend Engineer, you’ll join our team at an early stage and help shape key... ...to-end solutions What you bring: ~6+ years in backend software development ~ Strong proficiency in Kotlin or Java ~...SeniorTempo pienoStage/TirocinioContratto con partita IVARemotoOrario flessibile
- ...from EVs, heat pumps, and home batteries. The missing piece is software that can connect and orchestrate these devices at scale. Enode... ...energy transition. What we’re looking for: A Software Engineer to build and grow Enode’s Connect platform, which underpins how...Tempo pienoStage/TirocinioRemotoTurniOrario flessibile
Ricerche correlate
- ingegnere informatico da casa Smart working
- senior software engineer Smart working
- ingegnere informatico Smart working
- progettista software Smart working
- software developer senior Smart working
- software engineer Smart working
- sviluppatore informatico Smart working
- software engineer job part time Smart working
- programmatore softwarista Smart working
- programmatore software Smart working