Lead Observability & Data Architect / Engineer (SC / DV Cleared)
GIOS Technology, Blythswood New Town, Glasgow City
Lead Observability & Data Architect / Engineer (SC / DV Cleared)
Salary not available. View on company website.
GIOS Technology, Blythswood New Town, Glasgow City
- Full time
- Permanent
- Onsite working
Posted 4 days ago, 8 Jul | Get your application in now to be included in the first week's applications.
Closing date: Closing date not specified
Job ref: 4a8a65951a8d4e3c9afd5fefc8542814
Location ref: Blythswood New Town, Glasgow City
Full Job Description
- From Big Data/ETL to Telemetry Pipelines: If you have built high-throughput streaming pipelines using Apache Kafka, Airflow, NiFi, or Spark, you already know how to route and transform complex data feeds at scale.
- From Enterprise Search to System Observability: If you have engineered core search, indexing, or sharding architectures using Apache Solr, Lucene, or OpenSearch, you possess the exact foundational knowledge needed to scale an enterprise Elasticsearch cluster.
- From Cloud Infrastructure to Monitoring Fabrics: If you are a seasoned Cloud or Infrastructure Architect specializing in secure Kubernetes orchestration, Terraform automation, and air-gapped system design, you can easily pivot to defining secure logging architectures.
We are seeking an Enterprise Platform Architect or Senior Data Engineer to design and scale national-critical monitoring fabrics and data hubs. Whether your background is rooted in core Observability platforms (Opensearch/Elastic/Splunk/OpenTelemetry) or high-scale Data Engineering (Kafka, Python, Airflow, OpenSearch/Lucene), your core mission is the same: transforming billions of raw event logs into actionable, secure streaming telemetry. - Should be able to build high-performance data pipelines to ingest, parse, and structure massive event telemetry across hybrid-cloud and air-gapped environments.
- Manage large-scale cluster deployments, designing optimized indexing, data life cycle policies, and schema mapping (such as ECS).
- Expertise in Elasticsearch/OpenSearch, Splunk, or Solr. Strong proficiency in building visualization dashboards (Kibana, Grafana) and managing telemetry collectors (OpenTelemetry/OTel, Beats, Prometheus).
- Experience in designing and scaling data pipelines for high-velocity telemetry. Essential expertise in Apache Kafka, Logstash, or streaming/orchestration frameworks (Airflow, Vector, Debezium).
- Hands-on experience architecting scalable, containerized solutions using Kubernetes (K8s), Docker, and infrastructure-as-code (Terraform/Ansible).
- Search & Indexing Engine Depth: Prior architecture experience with OpenSearch, Apache Solr, or Lucene core or Elasticsearch.
- Strong proficiency in Python, Rust, or Bash for automating log parsing, testing, and deployment workflows.
- A strong track record of engineering within highly regulated sectors (eg, MoD, Home Office, BAE, HMRC) with an active UK security clearance (SC/DV).