Building and Scaling Engineering Teams

Scale engineering teams from 5 to 50+ without losing velocity, culture, or quality. Covers team topology design, hiring pipelines, onboarding programs, communication structures, and the organizational inflection points that require deliberate redesign.

Engineering teams grow from small, agile units into complex, multi-tiered organisations. This page guides engineering leaders through the practical, ongoing work of building and scaling teams—what to do, when, and how, with precision. It applies when a team transitions from 5 to 15 engineers, then scales to 50+, and when new roles, processes, and structures must evolve in tandem with technical complexity. It covers the day-to-day decisions, hidden pitfalls, and critical signals that determine whether a team remains effective, aligned, and sustainable as it grows.

Onboarding and Initial Integration

Onboarding is not a checklist but a system of feedback loops, starting the moment a new engineer accepts an offer. The first 90 days must produce not just a working laptop, but a sense of ownership and belonging.

On day one, the engineer receives a onboarding.yml file in their project’s docs/ directory:

onboarding:
  - step: "Set up local environment"
    command: "make setup"
    expected_output: "✅ All services running: api, web, worker, db"
    dependencies: ["docker", "docker-compose", "node", "python3"]
    required: true

  - step: "Run the first end-to-end test"
    command: "npm run test:e2e -- --spec=login.feature"
    expected_output: "✔ Test passed: User can log in and view dashboard"
    required: true

  - step: "Submit a PR to fix a known bug"
    command: "git checkout -b fix/missing-logging"
    expected_output: "✅ PR opened with description, tests passing, CI green"
    required: true

  - step: "Attend the weekly sync"
    time: "Tuesday 10:00 AM UTC"
    platform: "Zoom"
    agenda_url: "https://example.com/agenda/2024-04-01"
    required: true

The make setup script runs docker-compose up -d, starts the database with psql -c "CREATE DATABASE myapp;", and seeds data via python3 scripts/seed.py. If any command fails, the onboarding process halts, and the new engineer is prompted to file a help-needed issue in the project’s issue tracker.

A common failure: the onboarding script assumes node is in PATH, but the engineer’s environment has node in a non-standard location. The npm run test:e2e command fails silently, producing no output, yet the onboarding system marks the step as complete. To catch this, all onboarding scripts emit a status.json file with exit code, stdout, stderr, and timestamps. This file is uploaded to a status table in the onboarding.db SQLite database, which is queried daily by the engineering leadership dashboard.

Role Definition and Career Progression

Scaling teams requires clear, shared understanding of what each role entails. A role is not a job description but a set of expectations, responsibilities, and deliverables.

Each role is defined in a roles/ directory with a role.yaml file:

role: "Senior Backend Engineer"
level: 4
responsibilities:
  - "Own end-to-end delivery of a service from specification to production"
  - "Lead a technical deep dive on a cross-cutting concern"
  - "Review all PRs in their domain"
  - "Mentor two junior engineers"
expected_deliverables:
  - "One major feature shipped per quarter"
  - "One architecture decision recorded per quarter"
  - "One internal training session delivered"
evaluation_criteria:
  - "Code review feedback within 24 hours"
  - "Feature shipped with automated tests and monitoring"
  - "Documentation updated and accessible"
  - "Feedback collected from peers and stakeholders"

Promotion decisions are made via a promotion.yaml file submitted by the engineer’s manager and peers, containing:

promotion:
  engineer: "[email protected]"
  role: "Senior Backend Engineer"
  evidence:
    - "PR #123: Added caching layer to user service"
    - "ADR-004: Chosen Redis over Kafka for session management"
    - "Recorded training: 'Debugging Production Latency'"
  feedback:
    - author: "[email protected]"
      comment: "Alice led the team through a postmortem after a major outage"
      date: "2024-04-05"
    - author: "[email protected]"
      comment: "Her code reviews consistently reduce bug rates by 30%"
      date: "2024-04-08"

The promotion review is evaluated against the evaluation_criteria in the role.yaml. If any criterion is unmet, the reviewer must provide a concrete example or artifact. The system flags any missing evidence with a missing-evidence.json file attached to the review, containing links to the relevant PRs, ADRs, and recordings.

Structuring Teams for Scale

As teams grow beyond 15 engineers, they must evolve from monolithic units into autonomous, cross-functional squads. This requires decisions about team boundaries, ownership, and communication.

A team is defined by a team.yaml file in the project root:

team:
  name: "Payments"
  size: 12
  domain: "Financial transactions"
  ownership:
    - "End-to-end delivery of payment processing"
    - "Monitoring and alerting for all payment-related services"
    - "Onboarding new engineers to the Payments domain"
  responsibilities:
    - "Design and implement new payment methods"
    - "Maintain backward compatibility for legacy integrations"
    - "Coordinate with the Risk and Compliance teams"
  communication:
    - "Daily sync at 08:30 UTC"
    - "Biweekly planning session"
    - "Shared Slack channel: #payments"
    - "Quarterly demo to the entire engineering org"

The team’s success is measured by a health-check.json file updated weekly by a CI job:

{
  "team": "Payments",
  "check": "2024-04-15",
  "metrics": {
    "PR_throughput": 12,
    "PR_review_time": "2.1 days",
    "deployment_frequency": "3.5 per week",
    "mean_time_to_recover": "1.8 hours",
    "incident_rate": "1.2 per month"
  },
  "feedback": [
    {
      "source": "peer_review",
      "date": "2024-04-10",
      "comment": "Team is reactive but needs more proactive monitoring"
    },
    {
      "source": "engineering_retrospective",
      "date": "2024-04-08",
      "comment": "Too many overlapping responsibilities with the Checkout team"
    }
  ]
}

When the mean_time_to_recover exceeds 3 hours for three consecutive weeks, the team triggers a postmortem.yaml file in the docs/ directory:

postmortem:
  incident: "Payment processing failed during Black Friday peak"
  start_time: "2024-04-12T10:15:00Z"
  end_time: "2024-04-12T13:45:00Z"
  root_cause: "Database replication lag due to unindexed query in order creation"
  impact: "23,000 transactions lost, $1.2M in revenue"
  actions:
    - "Add index on orders.created_at"
    - "Implement alert for replication lag > 10s"
    - "Schedule monthly cross-team sync with DB team"
  owner: "[email protected]"
  status: "in_progress"

This postmortem is linked to the incident in the incident_log table in the monitoring.db database, which is queried by the alerting service every 15 minutes.

Managing Technical Debt and Architecture Evolution

As teams scale, technical debt accumulates and architecture evolves. Without a system to track and govern it, the team becomes reactive rather than proactive.

Each architectural decision is captured as an ADR (Architecture Decision Record), a single file in the adr/ directory:



Date: 2024-04-01

Status: Accepted

Context:
  We are building a financial system with complex transactional workloads.
  MySQL has been our primary database, but we need support for:
  - JSON column types
  - Row-level locking
  - Advanced indexing (partial, GIN, BRIN)

Decision:
  Adopt PostgreSQL as the primary database for the core transaction system.

Consequences:
  - Higher memory usage due to better indexing
  - Required migration of existing data and schema
  - Need to train team on advanced SQL features
  - Improved query performance for complex analytics

The adr/ directory contains a list.json file that aggregates all ADRs:

[
  {
    "id": "ADR-001",
    "title": "Choose Kubernetes for orchestration",
    "date": "2023-08-10",
    "status": "accepted"
  },
  {
    "id": "ADR-005",
    "title": "Choose PostgreSQL over MySQL",
    "date": "2024-04-01",
    "status": "accepted"
  }
]

A CI job runs scripts/check-adr-consistency.py weekly, which checks that every ADR has a status field, a date, and a consequences section. If any ADR is missing, it emits an error:

ERROR: ADR-005 is missing 'consequences' section
ERROR: ADR-002 has invalid date format: '2023-08-10T00:00:00Z' (expected YYYY-MM-DD)

The system also tracks technical debt through a debt.yaml file, which lists all known debt items with their status and priority:

technical_debt:
  - id: "TD-001"
    title: "Refactor legacy billing service"
    status: "in_progress"
    priority: "high"
    estimated_effort: "12 person-days"
    assigned_to: "[email protected]"
    due_date: "2024-06-30"
    related_to:
      - "ADR-003"
      - "PR-201"
    notes: "Service is monolithic and hard to deploy independently"

  - id: "TD-002"
    title: "Add centralized logging for all services"
    status: "planned"
    priority: "medium"
    estimated_effort: "8 person-days"
    assigned_to: "[email protected]"
    due_date: "2024-05-15"

When a team has more than five technical debt items in the in_progress state, the engineering leadership sends a team-health-alert to the team’s Slack channel, including a debt-dashboard.html file generated by scripts/generate-debt-dashboard.py.

Sustaining Culture and Collaboration

Culture is not a document. It is the sum of daily practices, rituals, and artefacts that form the team’s rhythm.

The team’s culture.yaml file defines the core practices:

culture:
  rituals:
    - name: "Weekly Standup"
      frequency: "weekly"
      time: "09:00 UTC"
      platform: "Zoom"
      agenda:
        - "What did you do last week?"
        - "What will you do this week?"
        - "Blockers and support needs"
      artifacts:
        - "Standup notes in a shared Google Doc"
        - "Action items logged in issue tracker"
    - name: "Monthly Tech Talk"
      frequency: "monthly"
      time: "14:00 UTC"
      platform: "Zoom"
      format:
        - "20-minute talk"
        - "10-minute Q&A"
        - "Recording archived in YouTube"
      artifacts:
        - "Talk slides in a public GitHub repository"
        - "Transcript in a Markdown file"
    - name: "Quarterly Retrospective"
      frequency: "quarterly"
      time: "10:00 UTC"
      platform: "Miro"
      format:
        - "Start with a timeline of the quarter"
        - "Group feedback into themes"
        - "Prioritize action items"
      artifacts:
        - "Miro board exported as PDF"
        - "Action items in the project’s `retrospectives/` directory"

The retrospectives/ directory contains retro-2024-01-01.json:

{
  "quarter": "Q1 2024",
  "date": "2024-01-01",
  "facilitator": "[email protected]",
  "participants": [
    "[email protected]",
    "[email protected]",
    "[email protected]"
  ],
  "themes": [
    {
      "title": "Communication",
      "notes": [
        "Team meetings are too long",
        "New members feel isolated",
        "Documentation is hard to find"
      ],
      "action_items": [
        {
          "task": "Shorten meetings to 30 minutes",
          "owner": "[email protected]",
          "due": "2024-03-15"
        },
        {
          "task": "Create a new onboarding video",
          "owner": "[email protected]",
          "due": "2024-04-01"
        }
      ]
    }
  ]
}

The system checks that every retrospective has at least three themes and that all action items have an owner and due date. If a retrospective is missing due dates, it logs a retro-missing-due-dates.log file with the list of missing items.

Continuous Learning and Feedback Loops

Scaling requires a feedback system that captures input from engineers, managers, and stakeholders.

A feedback-cycle.yaml file defines the process:

feedback_cycle:
  name: "Engineering Quarterly Review"
  frequency: "quarterly"
  phases:
    - phase: "Collect"
      duration: "1 week"
      tools:
        - "Survey via Google Forms"
        - "One-on-one interviews"
        - "Team retrospectives"
      deliverables:
        - "Survey responses in a shared spreadsheet"
        - "Interview transcripts"
        - "Summary report"
    - phase: "Synthesize"
      duration: "1 week"
      tools:
        - "Miro for thematic analysis"
        - "Google Docs for synthesis"
      deliverables:
        - "Key findings document"
        - "Themes and insights"
    - phase: "Act"
      duration: "2 weeks"
      tools:
        - "PRs for changes"
        - "Roadmap updates"
        - "Team meetings"
      deliverables:
        - "Updated team charter"
        - "New onboarding process"
        - "Investment in new tooling"

Each phase outputs a phase-report.json file:

{
  "phase": "Collect",
  "start_date": "2024-04-01",
  "end_date": "2024-04-08",
  "participants": [
    "[email protected]",
    "[email protected]",
    "[email protected]",
    "[email protected]"
  ],
  "data_sources": [
    "https://forms.gle/example1",
    "https://drive.google.com/file/d/12345"
  ],
  "key_findings": [
    "Engineers want more autonomy in their projects",
    "Time for learning is insufficient",
    "Onboarding takes too long"
  ]
}

The system monitors for missing deliverables, sending alerts when a phase is overdue. A missing summary report triggers a phase-overdue.json file with the list of missing items and a link to the project’s feedback-cycle/ directory.

Scaling the Leadership Layer

As teams grow, leadership must evolve from individual contributors to dedicated managers and architects.

The leadership team maintains a leadership.yaml file:

leadership:
  roles:
    - name: "Engineering Manager"
      responsibilities:
        - "Manage team performance and well-being"
        - "Support career progression"
        - "Coordinate cross-team initiatives"
        - "Conduct regular 1:1s"
      tools:
        - "1:1 template in Google Docs"
        - "Performance review cycle"
        - "Team health dashboard"
    - name: "Technical Lead"
      responsibilities:
        - "Guide technical direction"
        - "Lead architecture reviews"
        - "Mentor engineers"
        - "Drive adoption of best practices"
      tools:
        - "Architecture review board"
        - "Code review guidelines"
        - "Internal tech blog"
  meetings:
    - name: "Leadership Sync"
      frequency: "biweekly"
      time: "14:00 UTC"
      agenda:
        - "Review team health metrics"
        - "Discuss cross-team challenges"
        - "Plan for upcoming initiatives"
      artifacts:
        - "Meeting notes in a shared Notion page"
        - "Action items in the project’s `meetings/` directory"

The meetings/ directory contains leadership-sync-2024-04-10.json:

{
  "meeting": "Leadership Sync",
  "date": "2024-04-10",
  "agenda": [
    "Review team health metrics",
    "Discuss cross-team challenges",
    "Plan for upcoming initiatives"
  ],
  "decisions": [
    {
      "topic": "Adopt new monitoring tool",
      "decision": "Choose Prometheus over Grafana",
      "owner": "[email protected]",
      "status": "approved"
    },
    {
      "topic": "Expand onboarding to 90 days",
      "decision": "Add a second onboarding milestone at day 45",
      "owner": "[email protected]",
      "status": "in_review"
    }
  ]
}

When a decision is not approved, the system sends a decision-status-changed notification to the owner and the leadership team.

This page was rewritten on 17 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.