Ways of Engineering

This foundational document outlines our collective approach to engineering practices, fostering a culture of continuous improvement, innovation, and excellence. It is a compass for our technical journey, guiding us to make informed decisions, embrace best practices, and explore new technological frontiers. This document aims to harmonise our engineering efforts, ensuring efficiency, quality, and sustainability in our projects.

Statuses:

• ✅ Accepted
• 🧪 Experimenting
• 💡 New idea (waiting to be discussed by the team)

Hosting Services and Infrastructure

✅ Use GitHub as the Central Code Hosting and Collaboration Platform

We use GitHub as the central repository for our codebase, documentation, and collaboration. GitHub provides a reliable and widely adopted ecosystem for version control, collaboration, and CI/CD integration. Our team benefits from features such as pull requests, code reviews, issue tracking, and GitHub Actions, which streamline our development workflow. By leveraging GitHub, we ensure that our codebase remains accessible, maintainable, and transparent across the team.

✅ Use GitHub Services for Lightweight Infrastructure Needs

For projects that require lightweight infrastructure, we should use GitHub Services such as GitHub Pages and GitHub Actions. This reduces complexity for smaller projects, ensures fast delivery, and makes full use of GitHub’s integrated ecosystem while avoiding unnecessary cloud infrastructure for simple needs.

• GitHub Pages can be used to host static sites, documentation, or project portals with minimal overhead.
• GitHub Actions should be leveraged to automate workflows, such as CI/CD pipelines, testing, and simple infrastructure automation.

✅ Use Cloud Platform as the Primary Hosting Platform

The Cloud Platform is made for hosting modern, containerised applications and provides easy access to most AWS services for infrastructure requirements.

We should host new applications and infrastructure on the Cloud Platform, unless there are specific reasons not to.

✅ Use Modernisation Platform as the Secondary Hosting Platform

The Modernisation Platform is made for hosting applications that are not yet containerised or where a project has complex infrastructure requirements.

We have chosen to only have one environment* in the Modernisation Platform. This environment is called operations-engineering. AWS Accounts in this environment should be used for all infrastructure deemed suitable to host in the Modernisation Platform.

Any production level services should be deployed into the operations-engineering-production AWS Account. This account can only be deployed to via code changes in modernisation-platform-environments

**An environment in the Modernisation Platform can mean one of two things:*

• A namespace which encapsulates several AWS accounts i.e. operations-engineering
• A traditional environment for deployment such as development, production etc. combined with the Namespace definition above, these create the names for the AWS Accounts created i.e operations-engineering-development, operations-engineering-production etc.

In the descriptions in this section, I have opted to use environment to describe the namespace and AWS Account to describe the traditional environment to avoid confusion.

Why Are We Only Using One Environment?

While the team gets more familiar with the Modernisation Platform, we have decided to only use one environment. This will help us to keep things simple and avoid confusion.

We will review this decision in the future when there is a need to add more environments.

Cloud Platform vs Modernisation Platform

Cloud Platform	Modernisation Platform
Provides a namespace within the CP AWS account to contain your resources	Creates AWS accounts for each environment (dev, test, pre-prod, prod) under your namespace environment
Intended for microservices	Intended for services requiring more complex infrastructure
Access to some AWS resources restricted (eg no AWS Lambda)	Full control and full access to AWS tools
Provides access to CPs Kubernetes cluster	Provides Security baselines, Isolated networking, CI/CD for infrastructure
Example: Join GitHub app (🪦); containerised app hosting a website, running small supporting infra	Example: NOMIS; mainaining a huge database, restricting access, integrating with microservices (which may be on CP)

✅ Use Analytical Platform for Data Pipelines

The Analytical Platform is the recommended in-house service to support end-to-end data pipelines including raw data ingestion, transformation, data modelling, analysis, visualisation and dissemination as well as managing access permisions. Analytical Platform User Guidance.

✅ Do Not Use The MoJ Digital Services (DSD Account) to Host Services

The MoJ Digital Services AWS Account is a legacy account that is currently owned by several teams and is not suitable for hosting new services.

Existing services owned by our team hosted in the MoJ Digital Services AWS Account should be migrated to the Cloud Platform or Modernisation Platform.

✅ Terraform as IaC

We use Terraform as our Infrastructure as Code (IaC) tool to define, provision, and manage our infrastructure in an automated and consistent way. Instead of manually creating resources through cloud consoles, Terraform allows us to write infrastructure definitions in code, making the process repeatable, scalable, and version-controlled.

By using Terraform:

• We declare infrastructure in configuration files (HCL – HashiCorp Configuration Language).
• Terraform automatically provisions resources across cloud and on-premises providers.
• Execution plans show proposed changes before applying them, reducing risks.
• State management ensures our environments remain consistent and in sync.
• Version-controlled code enables collaboration, auditability, and rollback of changes.
• Modules allow us to reuse and standardize infrastructure components.

Package Management and Environment

✅ Python as the Primary Programming Language

We have decided to continue using Python as our primary programming language due to the following reasons:

• Python has a large and active community, ensuring abundant resources, libraries, and support.
• Python’s simplicity and readability make it easier to onboard new developers, especially those with an interest in infrastructure and DevOps.
• For our team’s requirements, Python’s flexibility and emphasis on rapid development suit our needs well. We prioritize the velocity of features over extreme performance or resilience.

✅ Use PipEnv for Python Package Manager

We are experimenting with adopting PipEnv as our Python package manager, moving away from our current setup using Pip. PipEnv offers a more streamlined approach to managing package dependencies and Python environments. This tool combines the best aspects of various packaging tools into a single interface, aiming to improve our workflow and project consistency. The experiment will assess PipEnv’s impact on our development process, focusing on ease of use, dependency resolution, and environment management.

Security and Dependency Management

✅ Replace Dependabot with Trivy for CVE Dependency Management

To enhance our security posture and streamline the management of vulnerabilities within our dependencies, we are experimenting with replacing Dependabot with Trivy. This shift aims to leverage Trivy’s comprehensive vulnerability detection across multiple languages and package managers and ease of integration into our CI/CD pipelines. During this experimental phase, we will evaluate Trivy’s effectiveness in identifying and mitigating known vulnerabilities, comparing it to Dependabot’s performance and utility.

✅ Pin GitHub Actions to a Commit Hash

Using a fixed version (via commit hash) ensures that you’re using exactly the version of an action you reviewed or tested. If you reference actions by a mutable tag like main or v1, the underlying code can change without warning, potentially introducing vulnerabilities, malicious code or breaking changes. Dependencies can evolve, sometimes breaking compatibility. If a new version of an action is released under the same tag, it might contain breaking changes that disrupt your workflow. Commit hashes lock in known-good versions.

✅ Team Hackathon

At least once a year we will run an in person Hackathon where we try to discover unintended ways of gaining access to our systems and data. This ensures we are testing for vulnerabilities and looking at the resilience of our existing security, enabling us to make changes and updates where necessary.

Adding/Adjusting Ways of Engineering

To maintain the relevance and effectiveness of our Ways of Engineering, we encourage team members to propose adjustments or additions. Suggestions can be discussed in dedicated Slack channels, Engineering Guild meetings, or directly through Pull Requests. Your contributions are crucial in ensuring our engineering practices remain dynamic, inclusive, and aligned with our goals.

Experimental Engineering Practices

We’re committed to innovation in our projects and how we work together as an engineering team. Experimenting with new practices allows us to improve and adapt to the changing tech landscape continuously. Participation and feedback from all team members are essential for these experiments to yield valuable insights and guide our evolution.

Principles and Values

Start Simple

We begin with a single Terraform configuration in a monorepo to keep things straightforward.

• A single root module is used for initial infrastructure (e.g., networking, compute, storage).
• This allows the team to quickly get started and deliver infrastructure without added complexity.
• State files are managed centrally (remote backend), ensuring team collaboration.

Add Complexity When Needed

As the infrastructure grows, we introduce modularity and structure to maintain scalability and reusability.

• Common infrastructure components (e.g., VPCs, databases, IAM roles, Kubernetes clusters) are abstracted into modules.
• Modules are version-controlled within the same repository, promoting reuse and standard practices.
• The monorepo continues to house all Terraform code, simplifying dependency management and visibility.

Monorepo Approach

We use a monorepo for Terraform code to ensure consistency and collaboration across teams.

• A single repository contains all environments (dev, staging, production) and their respective configurations.
• Shared modules are stored in a dedicated modules/ directory and referenced by environment-specific configurations.
• This enables a single source of truth for infrastructure, while also supporting CI/CD pipelines for automated validation and deployment.

Modules for Reusability and Standardization

• Modules encapsulate reusable patterns (e.g., network, eks, rds, iam).
• This reduces duplication, enforces organizational standards, and accelerates provisioning.
• Teams consume modules via versioned references, ensuring stability while allowing controlled updates.

Benefits of this Approach

• Simplicity First → Teams can start small without over-engineering.
• Scalability → As requirements grow, modular design supports complex infrastructure.
• Consistency → All environments follow the same structure and use shared modules.
• Collaboration → A monorepo ensures visibility, easier reviews, and central governance.
• Reusability → Modules allow faster delivery and standardization across projects.

This page was last reviewed on 11 August 2025. It needs to be reviewed again on 11 November 2025 by the page owner #operations-engineering-alerts .