Development Setup¶

How to get a working development environment for MANA, run the tests, and exercise the system at three levels of fidelity: unit tests with mocks, a Docker Compose stack, and a local Kubernetes cluster.

Prerequisites¶

You only need Python and Git for unit-test work.

1. Clone and Install¶

git clone https://github.com/othaime-en/mana.git
cd mana

python -m venv .venv
source .venv/bin/activate          # Windows: .venv\Scripts\activate
pip install -r requirements-dev.txt

One virtual environment serves both components. requirements-dev.txt at the repository root includes orchestrator/requirements-dev.txt (runtime packages plus pytest, pytest-cov, pytest-asyncio and httpx) and sample-app/requirements.txt. The two files must pin shared packages to the same version, and orchestrator/tests/test_requirements_alignment.py fails if they drift apart. When you bump a shared package such as pytest, change it in both files. CI and the Docker images still install each component's own file, so each runs with exactly its own dependencies.

Optional linters used by make lint: pip install flake8 black isort pylint.

2. Run the Tests¶

Tests mock Redis, Kubernetes and GitHub, so no cluster is needed.

make test                                   # both suites
cd sample-app   && pytest tests/ -v --cov=src --cov-report=term-missing
cd orchestrator && pytest tests/ -v --cov=src --cov-report=term-missing

Current baseline (at the time of writing):

CI runs both suites on every push and pull request, and fails when coverage drops below the gate. Raise the gate in .github/workflows/build-and-test.yml as coverage improves (the orchestrator target is 85%).

orchestrator/src/api.py creates the orchestrator at import time, which connects to Redis and loads a kubeconfig. Tests that import it must patch redis.Redis and the kubernetes.config/client calls (the api fixture in orchestrator/tests/conftest.py does this), or construct SelfHealingOrchestrator(use_config=False).

3. Run Services Locally Without Containers¶

Orchestrator¶

You need a Redis and a kubeconfig that points at any cluster (Kind is fine).

docker run -d --name mana-redis -p 6379:6379 redis:8-alpine
cd orchestrator
cp .env.example .env              # set GITHUB_TOKEN/GITHUB_REPO only if you want retries
set -a; source .env; set +a
uvicorn src.api:app --reload --port 8000

Open http://localhost:8000/docs for the interactive API. The orchestrator exits at startup if Redis or Kubernetes is unreachable.

Sample app¶

cd sample-app
FAILURE_RATE=0.1 ENVIRONMENT=development python -m flask --app src.app run --port 5000
curl localhost:5000/health

4. Docker Compose Stack¶

make dev-start        # docker compose up -d
make dev-logs
make dev-stop

This starts Redis, the sample app, the orchestrator, Prometheus, Grafana (http://localhost:3000, admin / admin123), Loki and Promtail.

!!! warning "Known Compose issues to be aware of" - The orchestrator service mounts ~/.kube/config to /root/.kube/config, but the image runs as the non-root user orchestrator, which cannot read /root. Mount the file to /home/orchestrator/.kube/config instead. - A kubeconfig for Kind or Minikube points at 127.0.0.1, which inside the container is the container itself. Use a kubeconfig whose server address is reachable from Docker (for Kind: kind get kubeconfig --internal with the Kind network attached). - The orchestrator service's healthcheck uses curl, which the orchestrator image does not install, so Compose reports it unhealthy. Use python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" or add curl to the image. - Prometheus in Compose expects an Alertmanager at alertmanager:9093, which Compose does not define; alert delivery will not work locally.

5. Local Kubernetes¶

kind create cluster --name self-healing-cicd        # or: minikube start
make build                                          # sample-app:<tag>, orchestrator:latest
make load-images                                    # kind/minikube image load
kubectl create namespace production
kubectl create namespace orchestrator
kubectl create secret generic github-token --from-literal=token=dummy -n orchestrator
kubectl apply -f orchestrator/k8s/ -n orchestrator
kubectl apply -f sample-app/k8s/ -n production
kubectl rollout status deployment/orchestrator -n orchestrator
make port-forward-orchestrator                      # in another terminal

Notes:

• make setup and make setup-cluster call scripts that are not in the repository yet; use the commands above.
• The sample-app Ingress requires an ingress controller; skip it with kubectl apply -f sample-app/k8s/deployment.yaml and delete the Ingress if you do not have one, or use port-forwarding.
• Alternatively ./scripts/deploy.sh --skip-monitoring does the whole sequence (see the CLI Reference); it needs Terraform and Helm.

Try the self-healing loop¶

export MANA=http://localhost:8000
# 1. Record a known-good version
curl -s -X POST $MANA/webhook/deployment -H 'Content-Type: application/json' \
  -d '{"deployment_id":"d1","namespace":"production","app_name":"sample-app","version":"1.0.0","status":"success"}'
# 2. Report a failure for a newer version
curl -s -X POST $MANA/webhook/deployment -H 'Content-Type: application/json' \
  -d '{"deployment_id":"d2","namespace":"production","app_name":"sample-app","version":"1.1.0","status":"failed","failure_type":"health_check_failure","metadata":{"workflow_run_id":"1"}}'
# 3. Inspect
curl -s $MANA/deployment/d2 | jq

Repeat step 2 with the same deployment_id to walk through retries and, after the fourth failure, the rollback. Retries need GITHUB_TOKEN/GITHUB_REPO to trigger a real re-run; without them the response still says retry, and the orchestrator logs the error.

6. Terraform¶

cd terraform
terraform fmt -recursive
terraform init -backend=false
terraform validate
./validate-terraform.sh
terraform plan -var="environment=local" -var="enable_staging=false" -var="enable_monitoring=false"

Use terraform init -backend=false for validation without touching state. State is local (terraform.tfstate); never commit it, since it can contain secrets.

7. Documentation¶

The docs are MkDocs with the shadcn theme and live in docs/; navigation is in mkdocs.yml.

pip install mkdocs mkdocs-shadcn pymdown-extensions
mkdocs serve                 # live preview at http://127.0.0.1:8000
mkdocs build --strict        # fails on broken links and nav errors

Admonitions (!!! note), tabs (=== "Title"), task lists and mermaid fences are enabled. Draw diagrams as SVG files in docs/assets/diagrams/ and embed them with a normal image link (for example ![Alt text](../assets/diagrams/component-map.svg)) instead of ASCII art.

Project Layout¶

orchestrator/   FastAPI service (src/, tests/, k8s/, Dockerfile)
sample-app/     Flask target app (src/, tests/, k8s/, Dockerfile)
terraform/      Modules: namespaces, sample-app, orchestrator, security, monitoring
kubernetes/     Standalone security manifests
monitoring/     Prometheus, Grafana, Loki, Promtail, Alertmanager configs
scripts/        deploy.sh
.github/        Workflows
docs/           This documentation

Editor and Quality Settings¶

• Format Python with black and sort imports with isort (make format). CI runs flake8 --max-line-length=100 --extend-ignore=E203,W503 and black --check (non-blocking today).
• Keep type hints on new functions.
• Add comments only where the code is not self-explanatory.

Troubleshooting¶