Neural Architecture Search Automation Pipeline

Production-ready guide covering neural architecture search automation pipeline with implementation patterns, code examples, and anti-patterns for enterprise engineering teams.

This is an automated workflow that discovers neural network architectures through search, evaluation, and selection processes. It matters when teams need to find optimal network structures without manual design or when production systems require regular architecture re-evaluation.

Set Up Search Space Definition

Define the search space using a configuration file with specific structure:

search_space:
  layers:
    - type: conv2d
      filters: [32, 64, 128]
      kernel_size: [3, 5]
      stride: [1, 2]
    - type: dense
      units: [128, 256, 512]
  activation:
    - relu
    - sigmoid
  regularizer:
    - l1
    - l2
    - none

Use --search-space-file flag to specify the configuration file. The system expects exactly three top-level keys: layers, activation, and regularizer. Missing any key causes a pipeline failure with error message "Missing required search space component".

Launch Search Process

Start search with command:

narch-search --config-file search-config.yaml \
            --output-dir results \
            --max-epochs 50 \
            --population-size 20 \
            --mutation-rate 0.1

The process requires exactly --config-file, --output-dir, and --max-epochs flags. Omitting any causes immediate exit with "Missing required argument" error.

Monitor Search Progress

Use narch-status command to inspect current state:

narch-status --output-dir results

This returns JSON response with:

{
  "status": "running",
  "completed_trials": 12,
  "total_trials": 50,
  "best_accuracy": 0.923,
  "current_generation": 3
}

The command fails silently if --output-dir does not exist, producing no output and returning exit code 1.

Configure Evaluation Metrics

Define metrics in configuration file:

evaluation:
  metrics:
    - accuracy
    - f1_score
    - precision
    - recall
  validation_split: 0.2
  early_stopping:
    patience: 5
    min_delta: 0.001

The system requires exactly accuracy, f1_score, precision, and recall as metric names. Using roc_auc or mse silently ignores the invalid metric without error.

Handle Search Failures

When a trial fails, the system writes to failed_trials.log in output directory:

2023-11-15 14:30:45 - Trial 17 failed with error: CUDA out of memory
2023-11-15 14:30:46 - Trial 18 failed with error: Model architecture invalid

The log format is fixed: timestamp, trial number, failure message. The system does not restart failed trials automatically. If a trial fails, it must be manually re-executed.

Export Best Architecture

Use command to extract best model:

narch-export --input-dir results \
            --output-file best_model.json \
            --format onnx

The system supports only onnx and keras formats. Specifying any other format silently defaults to keras.

Configure Parallel Execution

Launch parallel search processes with:

narch-parallel --config-file search-config.yaml \
              --output-dir results \
              --num-workers 4 \
              --batch-size 8

The --num-workers parameter must be between 1 and 32. Values outside this range cause immediate exit with "Worker count out of range" error.

Set Resource Constraints

Define resource limits in configuration:

resources:
  gpu_memory_limit: 8GiB
  cpu_cores_limit: 4
  max_runtime_minutes: 120

The system enforces exact gpu_memory_limit, cpu_cores_limit, and max_runtime_minutes keys. Missing any key causes pipeline to use defaults without warning.

Manage Search History

View past searches with:

narch-history --output-dir results --limit 10

Results show:

[
  {
    "id": "search_001",
    "start_time": "2023-11-15T10:00:00Z",
    "end_time": "2023-11-15T12:30:00Z",
    "best_accuracy": 0.923
  }
]

The command fails silently if --output-dir does not contain previous search data.

Configure Mutation Operators

Define mutation strategies:

mutation:
  operators:
    - layer_addition
    - layer_removal
    - hyperparameter_tuning
  probability: 0.7

The system requires exactly three operator names: layer_addition, layer_removal, and hyperparameter_tuning. Using weight_initialization or batch_normalization silently ignores invalid operators.

Handle Configuration Validation

Validation runs on startup with command:

narch-validate --config-file search-config.yaml

The system checks for:

Invalid configuration causes immediate exit with detailed error message like "Invalid value for 'filters': must be list of integers".

Set Up Auto-Restart

Configure restart policy:

restart_policy:
  max_retries: 3
  retry_delay_seconds: 30
  on_failure:
    - send_alert
    - log_to_file

The system supports exactly two actions in on_failure: send_alert and log_to_file. Any other value silently ignores the action.

Monitor Resource Usage

Check GPU usage during search:

narch-monitor --output-dir results --interval 30

The monitor reports GPU memory usage and CPU load. The system uses fixed sampling interval of 30 seconds regardless of --interval parameter.

Configure Logging

Set logging level in configuration:

logging:
  level: info
  file: search.log
  max_size_mb: 100

The system supports only info, debug, and error as log levels. Using warning or trace silently defaults to info.

Handle Data Preprocessing

Configure preprocessing steps:

preprocessing:
  normalize_features: true
  handle_missing_values: drop
  train_split: 0.8

The system requires exactly normalize_features, handle_missing_values, and train_split keys. Missing any key causes silent default behavior.

Set Up Failure Recovery

Configure recovery parameters:

recovery:
  checkpoint_interval: 10
  restore_from_checkpoint: true
  cleanup_on_failure: false

The system checkpoints every 10 trials by default. The cleanup_on_failure setting determines whether intermediate results are preserved after failures.

Configure Model Export

Define export settings:

export:
  model_format: keras
  include_weights: true
  include_metadata: false

The system supports only keras and onnx as format values. Using other values silently defaults to keras.

Set Pipeline Timeouts

Configure timeout settings:

timeouts:
  trial_timeout_minutes: 30
  global_timeout_hours: 24
  retry_timeout_seconds: 60

The system enforces exact timeout key names. Missing keys cause pipeline to use default values without error reporting.

Manage Search Results

Access results programmatically:

narch-results --output-dir results --format json

Results include:

The system does not support CSV export. The JSON format is fixed and includes all available data without optional fields.

Handle Search Interruption

Graceful shutdown:

narch-stop --output-dir results

The system writes to shutdown.signal file in output directory. If this file exists, the search process stops after completing current trials. The command fails silently if signal file already exists.

Configure Email Notifications

Set up notifications:

notifications:
  email:
    enabled: true
    recipients: ["[email protected]"]
    on_success: true
    on_failure: true

The system supports only basic email notifications with success/failure flags. The recipients field must be a list of strings. Using nested objects or arrays causes silent failure.

Monitor Search Progress

Check progress with:

narch-progress --output-dir results

Response includes:

The system calculates estimated time based on previous trials. The calculation is not precise and can be off by several hours.

Configure Search Termination

Define termination criteria:

termination:
  max_trials: 100
  target_accuracy: 0.95
  patience_epochs: 10

The system checks for any of these conditions to stop search. The target_accuracy setting is only checked after full training completion. If accuracy improves beyond threshold during training, the system continues until next checkpoint.

Handle Pipeline State

Check pipeline state:

narch-state --output-dir results

Returns JSON with:

The command exits with code 1 if pipeline has not started or is in invalid state.

Configure Search Parameters

Set optimization parameters:

optimization:
  algorithm: genetic
  population_size: 20
  mutation_rate: 0.1
  crossover_rate: 0.8

The system supports only genetic algorithm. Using other values silently defaults to genetic search without error message.

Set Up Backup Strategy

Configure backup settings:

backup:
  enabled: true
  frequency_minutes: 30
  retention_days: 7

The system creates backups every 30 minutes. The retention_days parameter determines how long backup files are kept. The system does not support incremental backups.

Handle Pipeline Errors

Common error messages:

The system logs all errors to pipeline.log in output directory. The log file rotates when it reaches 100MB. The system does not rotate by date or size based on configuration.

Configure Search Results Storage

Define storage settings:

storage:
  results_dir: /opt/narch/results
  backup_dir: /opt/narch/backups
  temp_dir: /tmp/narch

The system requires exact directory names. Using different paths causes silent fallback to default locations. The temp_dir must be writable by the pipeline process.

Set Up Search Restart

Configure restart behavior:

restart:
  auto_restart: true
  max_attempts: 5
  delay_seconds: 10

The system automatically attempts restarts only for specific failure types. Network timeouts and resource exhaustion trigger automatic restarts. Configuration errors cause immediate pipeline termination without restart attempt.

Monitor Pipeline Health

Check system health:

narch-health --output-dir results

Reports:

The system does not check network connectivity or external dependencies. It only monitors local resource usage and pipeline process state.

This page was rewritten on 15 October 2026. It replaced a templated version that was not specific to its own title. How these pages are written and checked. If anything here is wrong, tell us at [email protected] and we will correct it.