Quick Start Guide¶
This walkthrough runs MANA on a local Kind (or Minikube) cluster, deploys the bundled sample app, and then simulates a failing deployment so you can watch the orchestrator alert, retry, and finally roll back. It takes about 10 minutes and needs no GitHub account.
Prerequisites¶
1. Create a Cluster and Build the Images¶
git clone https://github.com/othaime-en/mana.git
cd mana
kind create cluster --name mana-demo
kubectl cluster-info --context kind-mana-demo
docker build -t orchestrator:latest orchestrator/
docker build -t sample-app:1.0.0 sample-app/
kind load docker-image orchestrator:latest sample-app:1.0.0 --name mana-demo
Using Minikube instead
bash
minikube start
docker build -t orchestrator:latest orchestrator/
docker build -t sample-app:1.0.0 sample-app/
minikube image load orchestrator:latest
minikube image load sample-app:1.0.0
Everything else in this guide is the same.
2. Install MANA and the Sample App¶
kubectl create namespace orchestrator
kubectl create namespace production
# MANA needs this secret to start. The value only matters for real GitHub retries.
kubectl create secret generic github-token \
--from-literal=token=placeholder -n orchestrator
kubectl apply -f orchestrator/k8s/redis.yaml
kubectl apply -f orchestrator/k8s/deployment.yaml
kubectl apply -f sample-app/k8s/ -n production
kubectl rollout status deployment/orchestrator -n orchestrator --timeout=180s
kubectl rollout status deployment/sample-app -n production --timeout=180s
The orchestrator waits 60 seconds before its first probe, so allow a minute or two. If the sample app's Ingress or HPA warn about a missing controller or metrics server, you can ignore them for this demo.
3. Check That It Works¶
kubectl -n orchestrator port-forward svc/orchestrator-service 8000:8000 &
export MANA=http://localhost:8000
curl -s $MANA/health | jq
{
"status": "healthy",
"redis": "healthy",
"kubernetes": "healthy",
"github_integration": true
}
(github_integration is true because the manifest sets both GITHUB_TOKEN and GITHUB_REPO. The token is a placeholder, so any real re-run request to GitHub will be rejected; that is expected in this demo. See Orchestrator Configuration.)
Open http://localhost:8000/docs for interactive API docs, and check the sample app:
kubectl -n production get pods -l app=sample-app
kubectl -n production port-forward svc/sample-app-service 8080:80 &
curl -s localhost:8080/health | jq
4. Teach MANA a Known-Good Version¶
MANA rolls back to the last version that reported success. In a real pipeline the deploy workflow sends this webhook for you; here you send it by hand:
curl -s -X POST $MANA/webhook/deployment -H 'Content-Type: application/json' -d '{
"deployment_id": "demo-good",
"namespace": "production",
"app_name": "sample-app",
"version": "1.0.0",
"status": "success"
}' | jq
{
"status": "success",
"deployment_id": "demo-good",
"message": "Deployment successful - version saved for future rollbacks"
}
Confirm it was stored:
kubectl -n orchestrator exec deploy/redis -- redis-cli get last_successful:production:sample-app
# 1.0.0
5. Break the Deployment¶
Roll out an image that does not exist, as a bad release would:
kubectl -n production set image deployment/sample-app sample-app=sample-app:1.1.0-broken
kubectl -n production get pods -l app=sample-app
Some pods will show ErrImagePull or ImagePullBackOff. The rollout is stuck on the new version.
6. Report the Failure and Watch MANA Decide¶
A pipeline would send a failed webhook when its rollout times out. Send it for the same deployment_id four times, waiting a moment between calls, and watch the decision change. (Run the log command in another terminal.)
kubectl -n orchestrator logs -l app=orchestrator -f --tail=5
for i in 1 2 3 4; do
echo "--- failure #$i"
curl -s -X POST $MANA/webhook/deployment -H 'Content-Type: application/json' -d '{
"deployment_id": "demo-bad",
"namespace": "production",
"app_name": "sample-app",
"version": "1.1.0-broken",
"status": "failed",
"failure_type": "deployment_failure",
"metadata": {"workflow_run_id": "1"}
}' | jq -c .action_taken
sleep 3
done
Expected output:
--- failure #1
{"action":"retry","retry_count":1,"backoff_seconds":10.0,"workflow_run_id":"1","message":"Retry 1/3 scheduled in 10.0s"}
--- failure #2
{"action":"retry","retry_count":2,"backoff_seconds":20.0,"workflow_run_id":"1","message":"Retry 2/3 scheduled in 20.0s"}
--- failure #3
{"action":"retry","retry_count":3,"backoff_seconds":40.0,"workflow_run_id":"1","message":"Retry 3/3 scheduled in 40.0s"}
--- failure #4
{"action":"rollback","previous_version":"1.0.0","message":"Rolled back to version 1.0.0"}
What happened:
| Failure | Decision | Why |
|---|---|---|
| 1, 2, 3 | retry |
Within MAX_RETRIES=3; a workflow re-run is scheduled after 10 s, 20 s and 40 s of backoff. Because the token is a placeholder, GitHub rejects the re-run request (an error such as Failed to trigger re-run: 401 appears in the logs after the backoff), which is harmless for this demo |
| 4 | rollback |
retry_count (4) now exceeds MAX_RETRIES, a previous version (1.0.0) is known, and the failed version is the one running |
The fourth call takes a few seconds: the orchestrator patches the Deployment back to sample-app:1.0.0, waits for all replicas to be ready, and calls each pod's /health endpoint before answering.
7. Verify the Recovery¶
kubectl -n production get deployment sample-app \
-o jsonpath='{.spec.template.spec.containers[0].image}{"\n"}'
# sample-app:1.0.0
kubectl -n production get pods -l app=sample-app
curl -s $MANA/deployment/demo-bad | jq
{
"deployment_id": "demo-bad",
"namespace": "production",
"app_name": "sample-app",
"version": "1.1.0-broken",
"status": "rolled_back",
"retry_count": 4,
"failure_type": "deployment_failure",
"previous_version": "1.0.0",
"timestamp": 1760000123.45,
"metadata": { "workflow_run_id": "1" }
}
See the audit trail for the whole story:
kubectl -n orchestrator logs -l app=orchestrator | grep "AUDIT:"
You should see deployment_received, failure_detected, retry_initiated, rollback_initiated, health_check_passed and rollback_completed entries.
You can also trigger the same rollback yourself at any time:
curl -s -X POST $MANA/rollback -H 'Content-Type: application/json' \
-d '{"namespace":"production","deployment_name":"sample-app","target_version":"1.0.0"}' | jq
Other Ways to Run Locally¶
Docker Compose¶
docker-compose.yml in the repository starts Redis, the sample app, the orchestrator and the monitoring stack (make dev-start). The orchestrator still needs a Kubernetes API, so this works only if the mounted kubeconfig is readable by the container's non-root user and points at an address reachable from Docker. See the notes in Development Setup. For a first try, the Kind route above is simpler.
Running the orchestrator on your machine¶
Run Redis in Docker and the orchestrator with uvicorn against your local kubeconfig, as described in Development Setup.
Clean Up¶
kill %1 %2 2>/dev/null # stop port-forwards started in this shell
kind delete cluster --name mana-demo
# or: minikube delete
Where to Go Next¶
- Installation: installing on a real cluster
- CI/CD Integration: have your pipeline send these webhooks automatically
- Configuration: tune retries, backoff and health checks
- Basic Usage: day-to-day operation
- Architecture Overview: how it all fits together