orca.autoscaler =============== .. py:module:: orca.autoscaler .. autoapi-nested-parse:: Custom Celery autoscaler for the OVRO-LWA Cluster. Provides a custom autoscaling strategy that considers system load average and available RAM when scaling worker processes. Classes ------- CalimScaler Autoscaler that scales workers based on load and memory constraints. Classes ------- .. autoapisummary:: orca.autoscaler.CalimScaler Module Contents --------------- .. py:class:: CalimScaler Bases: :py:obj:`celery.worker.autoscale.Autoscaler` Custom Celery autoscaler for resource-aware worker scaling. Extends the default Celery autoscaler to consider system load average and available RAM when making scaling decisions. This prevents over-provisioning on shared compute nodes. Scaling up occurs when: - More tasks are queued than processes available - System load average is below 20 - Available RAM exceeds 200 GB Scaling down occurs when: - Fewer tasks than processes, or load is acceptable .. note:: Cleanup and retry logic may need further implementation.