I’m @guyfromtheke 🇰🇪

Preamble This is my public open repository of how I’ve set up ( and continuing to setup up ) of Agro CD in my K8’s homelab. It also contains tips and tricks / commands that i have used to troubleshoot various issues. What is ArgoCD? ArgoCD is a declarative, GitOps continuous delivery tool for Kubernetes. It automates the deployment of applications and helps maintain their desired state by continuously monitoring your Git repositories and ensuring your Kubernetes cluster matches the defined state in your Git configuration. ...

Optimizing Resource Limits and Pod Distribution for ArgoCD Overview This guide details the process of optimizing an ArgoCD deployment in Kubernetes by implementing proper resource limits and ensuring optimal pod distribution across worker nodes. These optimizations enhance reliability, performance, and high availability of the ArgoCD installation. Initial Environment Analysis Cluster Configuration Kubernetes Cluster (K3s v1.31.6+k3s1) Architecture: 1 master node (control plane) 2 worker nodes Initial state: Pods distributed across all nodes including master Master Node Protection Implementing Control Plane Isolation # Adding taint to master node kubectl taint nodes <master-node> node-role.kubernetes.io/control-plane=:NoSchedule Purpose of Master Node Taint Prevents regular workloads from scheduling on the control plane Reserves master node resources for critical cluster operations Enforces best practices for Kubernetes architecture Resource Limits Configuration Resource Allocation by Component Application Controller resources: requests: cpu: 100m memory: 128Mi limits: cpu: 500m memory: 512Mi API Server resources: requests: cpu: 50m memory: 64Mi limits: cpu: 200m memory: 128Mi Repository Server resources: requests: cpu: 50m memory: 64Mi limits: cpu: 200m memory: 256Mi Dex Server resources: requests: cpu: 50m memory: 128Mi limits: cpu: 200m memory: 256Mi Redis resources: requests: cpu: 50m memory: 32Mi limits: cpu: 100m memory: 64Mi ApplicationSet Controller resources: requests: cpu: 50m memory: 32Mi limits: cpu: 200m memory: 128Mi Notifications Controller resources: requests: cpu: 50m memory: 32Mi limits: cpu: 100m memory: 128Mi Pod Distribution Strategy Anti-Affinity Rules affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 podAffinityTerm: labelSelector: matchExpressions: - key: app.kubernetes.io/name operator: In values: - <component-name> topologyKey: kubernetes.io/hostname Final Pod Distribution Worker Node 1: argocd-repo-server argocd-server Worker Node 2: argocd-application-controller argocd-applicationset-controller argocd-dex-server argocd-notifications-controller argocd-redis Optimization Results Resource Management Benefits Predictable resource usage Prevention of resource contention Efficient resource allocation Protection against memory/CPU exhaustion High Availability Improvements Proper workload distribution Enhanced fault tolerance Better resource utilization Reduced single point of failure risk Performance Metrics CPU usage maintained within defined limits Memory consumption optimized Improved response times Better overall cluster stability Monitoring and Maintenance Health Checks # View pod distribution kubectl get pods -n argocd -o wide # Check resource usage kubectl top pods -n argocd Resource Adjustment Monitor resource usage and adjust limits based on: ...

ArgoCD Real-World Implementation Scenarios Multi-Cluster Management Setting Up Multi-Cluster Architecture apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: multi-cluster-app namespace: argocd spec: destination: server: https://kubernetes.default.svc namespace: production project: default source: path: kustomize/overlays/production repoURL: https://github.com/your-org/gitops targetRevision: HEAD syncPolicy: automated: prune: true selfHeal: true Cross-Cluster Synchronization Managing dependencies Cluster registration Resource propagation Network configuration Hybrid Cloud Deployments Cloud Provider Integration AWS EKS Configuration Azure AKS Setup GCP GKE Implementation On-premises Integration Network Considerations Cross-cloud connectivity Service mesh integration Load balancing Security groups Microservices Orchestration Application Definition apiVersion: argoproj.io/v1alpha1 kind: ApplicationSet metadata: name: microservices namespace: argocd spec: generators: - list: elements: - service: auth namespace: auth-system - service: payment namespace: payment-system template: metadata: name: '{{service}}' spec: project: microservices source: repoURL: https://github.com/your-org/{{service}} targetRevision: HEAD path: kubernetes destination: server: https://kubernetes.default.svc namespace: '{{namespace}}' Service Dependencies Dependency graphs Startup order Health checks Rollback strategies Production Best Practices High Availability Setup Multiple replicas Pod anti-affinity Network redundancy Backup strategies Scaling Strategies apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: application-hpa spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: my-app minReplicas: 3 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 Compliance and Security Audit Logging apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd data: application.resourceTrackingMethod: annotation resource.customizations: | argoproj.io/Application: health.lua: | hs = {} hs.status = "Progressing" hs.message = "" return hs Security Measures Network policies Pod security policies Secret management Access controls Disaster Recovery Backup Configuration # Backup ArgoCD state argocd admin export > argocd-backup.yaml # Backup application configurations kubectl get applications -n argocd -o yaml > applications-backup.yaml # Backup secrets kubectl get secrets -n argocd -o yaml > secrets-backup.yaml Recovery Procedures Infrastructure recovery Data restoration Application reconciliation Validation checks Performance Optimization Resource Management CPU/Memory tuning Storage optimization Network throughput Cache configuration Monitoring Setup apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: argocd-metrics spec: selector: matchLabels: app.kubernetes.io/name: argocd-metrics endpoints: - port: metrics Case Studies E-Commerce Platform Microservices architecture Blue-green deployments Canary releases A/B testing Financial Services Compliance requirements Security measures Audit trails Zero-downtime updates SaaS Application Multi-tenant setup Resource isolation Scalability patterns Monitoring solutions Troubleshooting Guide Common Issues Sync failures Resource conflicts Network problems Authentication issues Debug Procedures # Check application status argocd app get <app-name> # View detailed sync status argocd app sync <app-name> --debug # Check controller logs kubectl logs -n argocd -l app.kubernetes.io/name=argocd-application-controller Multi-Cluster Management Extended Guide Cluster Addition Workflow # Add new cluster to ArgoCD argocd cluster add <context-name> # Verify cluster connection argocd cluster list # Test cluster access argocd app list --dest-server <cluster-url> Multi-Cluster Sync Strategies Sequential Deployment apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: multi-cluster-app namespace: argocd annotations: argocd.argoproj.io/sync-wave: "1" # Control sync order spec: project: default source: repoURL: https://github.com/your-org/gitops path: kustomize/base targetRevision: HEAD destination: server: https://kubernetes.default.svc namespace: production Parallel Deployment apiVersion: argoproj.io/v1alpha1 kind: ApplicationSet metadata: name: parallel-deploy namespace: argocd spec: generators: - clusters: {} # Deploy to all connected clusters template: metadata: name: '{{name}}-app' spec: project: default source: repoURL: https://github.com/your-org/gitops path: kustomize/base destination: server: '{{server}}' namespace: production Cluster-Specific Configurations Environment Overrides apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: cluster-specific-app spec: source: repoURL: https://github.com/your-org/gitops path: kustomize/base kustomize: namePrefix: dev- commonLabels: environment: development Homelab-Specific Considerations Resource Optimization Limited Resources Setup apiVersion: v1 kind: ResourceQuota metadata: name: argocd-quota namespace: argocd spec: hard: requests.cpu: "2" requests.memory: 2Gi limits.cpu: "4" limits.memory: 4Gi Network Optimization apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: argocd-network-policy namespace: argocd spec: podSelector: matchLabels: app.kubernetes.io/part-of: argocd ingress: - from: - namespaceSelector: matchLabels: name: monitoring Local Storage Configuration Local Path Provisioner apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: local-path provisioner: rancher.io/local-path volumeBindingMode: WaitForFirstConsumer Advanced Use Cases Blue-Green Deployments apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: blue-green-app spec: source: plugin: name: blue-green repoURL: https://github.com/your-org/gitops syncPolicy: automated: prune: true selfHeal: true Canary Deployments apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: canary-app annotations: argocd-image-updater.argoproj.io/image-list: app=your-org/app:latest spec: source: repoURL: https://github.com/your-org/gitops path: kustomize/overlays/canary Integration Examples Monitoring Stack apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: monitoring-stack spec: source: repoURL: https://github.com/prometheus-community/helm-charts targetRevision: HEAD helm: values: | grafana: enabled: true prometheus: enabled: true CI/CD Pipeline Integration apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: jenkins-integration spec: source: repoURL: https://github.com/your-org/jenkins-config path: kubernetes directory: recurse: true Troubleshooting Advanced Scenarios Multi-Cluster Issues # Check cluster connectivity argocd cluster get <cluster-url> # Verify cluster resources kubectl --context=<cluster-context> get nodes # Test application deployment argocd app sync <app-name> --dest-server <cluster-url> Sync Issues # Debug sync problems argocd app sync <app-name> --debug # Check resource health argocd app get <app-name> --refresh # View detailed sync status argocd app history <app-name> Performance Problems # Monitor sync performance argocd app list --output wide # Check resource utilization kubectl top pods -n argocd # View detailed metrics argocd admin metrics Best Practices Security Measures Implement network policies Use RBAC effectively Rotate credentials regularly Enable audit logging High Availability Multiple replicas Pod anti-affinity Resource limits Backup strategies Monitoring Prometheus integration Grafana dashboards Alert configuration Log aggregation Next Steps Return to the main guide or explore Integration Patterns. ...

ArgoCD Integration Patterns CI/CD Pipeline Integration GitHub Actions Integration name: Deploy to ArgoCD on: push: branches: [ main ] jobs: deploy: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Install ArgoCD CLI run: | curl -sSL -o argocd https://github.com/argoproj/argo-cd/releases/latest/download/argocd-linux-amd64 sudo install -m 555 argocd /usr/local/bin/argocd - name: Deploy to ArgoCD run: | argocd app sync my-application --auth-token ${{ secrets.ARGOCD_TOKEN }} Jenkins Pipeline pipeline { agent any environment { ARGOCD_SERVER = 'argocd.example.com' ARGOCD_TOKEN = credentials('argocd-token') } stages { stage('Deploy') { steps { sh ''' argocd login $ARGOCD_SERVER --auth-token $ARGOCD_TOKEN --insecure argocd app sync my-application ''' } } } } GitLab CI Integration deploy: stage: deploy script: - argocd login $ARGOCD_SERVER --auth-token $ARGOCD_TOKEN - argocd app sync my-application only: - main External Tools Connectivity Prometheus Integration apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: argocd-metrics namespace: monitoring spec: selector: matchLabels: app.kubernetes.io/name: argocd-metrics endpoints: - port: metrics interval: 30s Grafana Dashboard apiVersion: integreatly.org/v1alpha1 kind: GrafanaDashboard metadata: name: argocd-dashboard spec: json: | { "dashboard": { "title": "ArgoCD Metrics", "panels": [ { "title": "Sync Status", "type": "graph" } ] } } Slack Notifications apiVersion: v1 kind: ConfigMap metadata: name: argocd-notifications-cm data: service.slack: | token: $slack-token username: ArgoCD trigger.on-sync-status-changed: | - send: [slack] Authentication Systems OIDC Configuration apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd data: url: https://argocd.example.com dex.config: | connectors: - type: oidc id: google name: Google config: issuer: https://accounts.google.com clientID: your-client-id clientSecret: $oidc-secret LDAP Integration apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm data: url: https://argocd.example.com dex.config: | connectors: - type: ldap name: ActiveDirectory id: ad config: host: ldap.example.com:389 bindDN: cn=admin,dc=example,dc=com bindPW: $ldap-password Git Provider Integration GitHub Integration apiVersion: v1 kind: Secret metadata: name: repo-secret namespace: argocd stringData: type: git url: https://github.com/organization/repo password: github-token username: git GitLab Integration apiVersion: v1 kind: Secret metadata: name: gitlab-secret namespace: argocd stringData: type: git url: https://gitlab.com/organization/repo password: gitlab-token username: git Custom Tool Integration Webhook Configuration apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm data: webhooks: | - type: GitHub endpoint: /api/webhook url: https://github.com/your-org/repo secret: $webhook-secret Custom Health Checks apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm data: resource.customizations: | custom.group/Kind: health.lua: | hs = {} if obj.status.health == "healthy" then hs.status = "Healthy" else hs.status = "Progressing" end return hs Monitoring Solutions Datadog Integration apiVersion: v1 kind: Secret metadata: name: datadog-secret namespace: argocd stringData: api-key: your-datadog-api-key --- apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm data: datadog.enabled: "true" datadog.address: "https://api.datadoghq.com" New Relic Integration apiVersion: monitoring.newrelic.com/v1alpha1 kind: NRQLAlert metadata: name: argocd-sync-alert spec: query: "SELECT count(*) FROM ArgoCD WHERE status = 'Failed'" threshold: 1 Security Integration Vault Integration apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm data: vault.enabled: "true" vault.addr: "https://vault.example.com" vault.role: "argocd" Certificate Management apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: argocd-cert spec: secretName: argocd-secret issuerRef: name: letsencrypt-prod kind: ClusterIssuer dnsNames: - argocd.example.com Troubleshooting Integrations Common Integration Issues Authentication failures Webhook misconfiguration Permission problems Network connectivity Debug Procedures # Check integration status argocd admin settings validate # Test webhook curl -X POST https://argocd/api/webhook -d @webhook-payload.json # Verify connections argocd admin app validate Best Practices Security Considerations Use service accounts Implement least privilege Regular secret rotation Audit logging Performance Optimization Rate limiting Cache configuration Connection pooling Resource allocation Homelab Integration Examples Local Git Server Integration (Gitea) apiVersion: v1 kind: Secret metadata: name: gitea-repo-secret namespace: argocd stringData: type: git url: http://gitea.local/your-org/repo username: git password: your-token Local Registry Integration apiVersion: v1 kind: Secret metadata: name: local-registry-secret namespace: argocd type: kubernetes.io/dockerconfigjson stringData: .dockerconfigjson: | { "auths": { "registry.local:5000": { "auth": "base64-encoded-auth" } } } Tailscale VPN Integration apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd data: timeout.reconciliation: 180s kustomize.buildOptions: --network HomeLab Monitoring Integration Prometheus Stack apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: prometheus-stack namespace: argocd spec: project: default source: repoURL: https://github.com/prometheus-community/helm-charts.git targetRevision: HEAD path: charts/kube-prometheus-stack helm: values: | grafana: adminPassword: your-admin-password prometheus: prometheusSpec: retention: 15d storageSpec: volumeClaimTemplate: spec: storageClassName: local-path resources: requests: storage: 50Gi Grafana Dashboard Configuration apiVersion: v1 kind: ConfigMap metadata: name: argocd-grafana-dashboard namespace: monitoring labels: grafana_dashboard: "true" data: argocd-dashboard.json: | { "title": "ArgoCD Overview", "panels": [ { "title": "Sync Status", "type": "gauge", "datasource": "Prometheus" } ] } Local Authentication Integration LDAP with FreeIPA apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd data: url: https://argocd.local dex.config: | connectors: - type: ldap name: FreeIPA id: freeipa config: host: freeipa.local:389 insecureNoSSL: false bindDN: uid=service-account,cn=users,cn=accounts,dc=local bindPW: $LDAP_PASSWORD userSearch: baseDN: cn=users,cn=accounts,dc=local filter: (objectClass=person) username: uid idAttr: uid emailAttr: mail nameAttr: displayName groupSearch: baseDN: cn=groups,cn=accounts,dc=local filter: (objectClass=groupOfNames) userAttr: DN groupAttr: member nameAttr: cn Advanced Integration Examples HashiCorp Vault Integration apiVersion: v1 kind: Secret metadata: name: argocd-vault-plugin-credentials namespace: argocd stringData: VAULT_ADDR: "http://vault.local:8200" VAULT_TOKEN: "your-vault-token" --- apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd data: configManagementPlugins: | - name: argocd-vault-plugin generate: command: ["argocd-vault-plugin"] args: ["generate", "./"] Custom Metrics Integration apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: argocd-metrics namespace: monitoring spec: selector: matchLabels: app.kubernetes.io/name: argocd-metrics endpoints: - port: metrics interval: 30s path: /metrics metricRelabelings: - sourceLabels: [__name__] regex: 'argocd_.*' action: keep Backup Integration with Minio apiVersion: v1 kind: Secret metadata: name: minio-backup-credentials namespace: argocd stringData: AWS_ACCESS_KEY_ID: your-access-key AWS_SECRET_ACCESS_KEY: your-secret-key --- apiVersion: batch/v1 kind: CronJob metadata: name: argocd-backup namespace: argocd spec: schedule: "0 1 * * *" jobTemplate: spec: template: spec: containers: - name: backup image: bitnami/kubectl command: - /bin/sh - -c - | argocd admin export > /backup/argocd-$(date +%Y%m%d).yaml mc cp /backup/* minio/argocd-backup/ Next Steps Return to the main guide or review Real-World Scenarios. ...

ArgoCD Quick Reference Guide Common Operations Installation # Install ArgoCD kubectl create namespace argocd kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml # Access the UI kubectl patch svc argocd-server -n argocd -p '{"spec": {"type": "NodePort"}}' # Get initial admin password kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d Application Management # Create new application argocd app create myapp \ --repo https://github.com/your-org/your-app.git \ --path kustomize \ --dest-server https://kubernetes.default.svc \ --dest-namespace default # Sync application argocd app sync myapp # Get application status argocd app get myapp # Delete application argocd app delete myapp User Management # Add cluster admin argocd account update-password \ --current-password <initial-password> \ --new-password <new-password> # Create project argocd proj create myproject \ --description "My new project" \ --src https://github.com/your-org/* \ --dest https://kubernetes.default.svc,default Quick Troubleshooting Health Checks # Check pod health kubectl get pods -n argocd # Check logs kubectl logs -n argocd -l app.kubernetes.io/name=argocd-server # Verify sync status argocd app get myapp --refresh Common Issues & Solutions Sync Failed Check application logs: argocd app logs myapp Verify Git credentials: argocd repo list Test repository access: argocd repo test https://github.com/your-org/your-app.git Access Issues Reset admin password: argocd admin initial-password -n argocd Check RBAC: kubectl get cm argocd-rbac-cm -n argocd -o yaml Quick Configurations SSL/TLS Setup apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: argocd-cert namespace: argocd spec: secretName: argocd-server-tls issuerRef: name: letsencrypt-prod kind: ClusterIssuer dnsNames: - argocd.your-domain.com Basic Auth Integration apiVersion: v1 kind: ConfigMap metadata: name: argocd-cm namespace: argocd data: accounts.admin: apiKey,login Environment-Specific Examples Development apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: dev-app spec: source: repoURL: https://github.com/your-org/your-app.git targetRevision: develop path: k8s/dev destination: server: https://kubernetes.default.svc namespace: development Production apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: prod-app spec: source: repoURL: https://github.com/your-org/your-app.git targetRevision: main path: k8s/prod destination: server: https://kubernetes.default.svc namespace: production syncPolicy: automated: prune: true selfHeal: true Network Configurations Ingress Example apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: argocd-ingress annotations: nginx.ingress.kubernetes.io/ssl-passthrough: "true" cert-manager.io/cluster-issuer: letsencrypt-prod spec: rules: - host: argocd.your-domain.com http: paths: - path: / pathType: Prefix backend: service: name: argocd-server port: number: 443 Network Policy apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: argocd-server-network-policy namespace: argocd spec: podSelector: matchLabels: app.kubernetes.io/name: argocd-server ingress: - from: - namespaceSelector: matchLabels: name: ingress-nginx Monitoring Quick Setup Prometheus Rules apiVersion: monitoring.coreos.com/v1 kind: PrometheusRule metadata: name: argocd-rules namespace: monitoring spec: groups: - name: argocd.rules rules: - alert: ArgoCDSyncFailed expr: argocd_app_sync_status{status="Failed"} > 0 for: 5m labels: severity: critical Next Steps For detailed information, refer to: ...