Data Engineer
Full-timeDescription
PixelPlex is currently looking for an experienced Data Engineer to join our new project. It is a Business Intelligence and Data Visualization service designed especially for NFT enthusiasts. Service uses on-chain, social and other metrics, enriches it to deliver unique insights ready for decisions.
main focus
- Data Quality: automating the DQ process using Great Expectations;
- Data Marts: presenting data marts, optimizing analyst code if necessary;
Ensuring the collection and data loading into analytical and hot databases, including obtaining structured and unstructured data from various sources, preparing, cleaning, and pre-processing data from external sources, and building aggregates;
responsibilities
- Creating and improving data processing processes (optimizing processor groups, optimizing data marts);
- Participating in the development of data processing solutions within ML projects and deploying pipelines;
- Developing and optimizing procedures for generating detailed data layers (raw to ods) and data mart layers in DataLake and hot database for online reporting;
- Implementing CI/CD processes and monitoring the developed data processing processes (Grafana, Prometheus);
- Handling ad-hoc queries.
requirements
- Knowledge of database principles and HD design;
- Experience in ETL process development;
- Good understanding of ML pipelines and their deployment to production;
- Experience using Airflow (or other industry-standard pipeline orchestrators, such as Luigi, Dagster, etc.);
- Experience with high-load distributed data storage and processing systems;
- Excellent knowledge of SQL, query optimization experience;
- Willingness to learn, grow, and be motivated to achieve goals.
nice to have
- Experience in Python or Node.js development;
- Knowledge of DevOps tools;
- Experience with cloud services.
- Services IT R&D RESEARCH & DEVELOPMENT In this ever-evolving tech realm, don't just follow trends — set them.Capitalize on innovation, launch ground-breaking products, and forge new business models for maximum ROI. IT Consulting CONSULTING From complex enterprise tech transformation to the innovative project launch, our team supports businesses at different stages of their projects.Come along, we’ll help you get an edge and play big on the global market. Custom Software Development CUSTOM ENGINEERING Let us have your back in a project of any scale. From user-centric mobile apps to full-blown cross-platform enterprise ecosystems — we’ll bring your concept to life, exactly as you think it should look and work. Mobile App Development MOBILE APP DEVELOPMENT Entrust us with your end-to-end mobile project — from ideation and engineering to app launch and integration.With business growth in mind, we’ll help you hit the market with a slick iOS, Android, or cross-platform app. Mobile App Development iOS App Development Android App Development Cross-Platform App DevelopmentWeb Development WEB DEVELOPMENT Whether you need an app from the ground up or require a legacy system to be updated, we can jump in at any stage.From an accessibility roadmap to post-launch support, we’ll help your business stay strictly legal and competitive. Web Development Web Accessibility Audit Web Accessibility Consulting Accessible Web Development and DesignDevSecOps DevSecOps Ensure your software's integrity and efficiency from conception to compliance. UI/UX Design UI/UX DESIGN Give us the pleasure of adding our secret sauce to your app.We’ll create beautiful screens at the front while breaking the limits of what’s behind them to help your app get to beyond-plausible business achievements. QA & Software Testing QA & SOFTWARE TESTING Engage us for integrated quality assurance services, and our experts will advise on QA strategy and optimize software testing costs.We’ll balance manual testing with QA automation to ensure consistent performance for all possible use cases and devices.
- Technologies Blockchain BLOCKCHAIN DEVELOPMENT Blockchain Development Blockchain Consulting NFT Development STO Development ZK Rollup Solutions Enterprise Blockchain Development Smart Contracts Development Smart Contract Audit DApp Development Cryptocurrency Exchange DevelopmentData Science DATA SCIENCE Transforming data into growth strategies is our specialty.Leverage our expertise to unlock the potential of your big data and diverse digital assets, driving business growth. Big Data Consulting Data Analytics Business Intelligence Data VisualizationMachine Learning and AI ARTIFICIAL INTELLIGENCE Machine Learning Predictive Analytics Computer Vision Custom AI Development Chatbot DevelopmentAR & VR AUGMENTED VIRTUAL REALITY Immerse your customers into a universe of unimaginable and give them truly novel experiences with AR, VR, and Mixed Reality.We'll help define a proper business concept and find a balance between legacy workflows and next-gen customer engagement solutions. Metaverse METAVERSE We create tools, assets, and ecosystems to seamlessly merge real-life and digital worlds within your Metaverse projects.It could be a multi-layer virtual space or a unique artwork item. Either way, we’ll deliver it — ready and working. Connected Devices (IoT) INTERNET-OF-THINGS Aching to handle digital and physical asset management? We build load-resistant IoT services, both enterprise and consumer.Hit us with IoT consulting, app development, back-end engineering, or existing infrastructure revamping – we’ll nail it down.
- Industries FinTech & Banking FINTECH & BANKING As traditional finance goes digital, we are committed to building efficient ecosystems and better engagementThink of customized FinTech solutions with tamper-proof transactions and storage, progress transparency and automation — and we’ll make them see the light of day. Retail & eCommerce RETAIL & ECOMMERCE Whether you market B2B or B2C, commerce tech trends are all about value-driven purposes, global sustainability, hybrid shopping journeys, and extra-resiliency.Let your clients know that there’s more to your brand than meets the eye by creating unique customer experiences in all your stores. Supply Chain & Logistics SUPPLY CHAIN & LOGISTICS To make things easier for all vendors, we deliver apps for route and cost optimization, vehicle operational support, and better dispatch time efficiency.With focus is sustainability, resilience, transparency, and immutability, let’s get your transformation going. Healthcare HEALTHCARE Custom healthcare software solutions are aimed at helping you ensure accurate diagnosis, better patient engagement, and positive healthcare outcomes.Whether you require a patient management solution, practice management software, EMR/EHR system, or ML-enabled diagnostics – we’ve got you covered. Real Estate REAL ESTATE Keep up with digital innovation trends by accelerating enterprise transformation and scaling, leveraging data and orchestrating workflows.Whether you manage and sell commercial facilities or invest third-party capital, our integrated solutions help you make the most of it. Oil & Gas OIL & GAS With mobility and digital technologies standing to change the game and define leadership, our mission is to get you digital-first.Resolve operational and conceptual issues by introducing clear tech vision, feasible architectures, and flexible software to take business extension off limits.
- Solutions Know-Your-Transaction KYT platform ensures financial integrity, compliance, and proactive risk management. Control your operations seamlessly with real-time API integration. DocFlow Secure workflows, decentralized storage, and total document control. Streamline your processes with our business document management system. Arbitrage Bot Alternate your yield channels, maximize investment efficiency, customize trading strategies, and execute profitable low-risk transactions. CryptoAPI Connect your dApps to blockchains commission-free. No need for heavy database upload, infrastructure updates, or a dedicated engineering team. OTC Hawk Redefine wealth and portfolio management at your firm, outpacing your competitors. Make crypto/fiat trading easy and profitable for your clients.
- Success stories Blockchain Government & Enterprise Energy & Utilities Financial Services Supply Chain Healthcare Retail & eCommerce See all projects
- Company About us Team News Careers Scholarship CSR Contacts
- Blog