ACE Components

The goal of this design document is to describe logical architecture of how ACE components interact with each other.

ACE can be deployed in 2 modes - Production and Development.

Production mode

Production mode is intended for production use and exposes only user defined Dynamic APIs. All ACE client requests are handled by runtime serve and deployment is done by deployment API server either through CLI or API.

Development mode

Development mode is intended for design and development. Runtime server in development mode exposes additional development APIs which are intended for development use only.

Service sizing

Please keep in mind that Sizing includes only resources needed by ACE components and do not include sizing for accommodating infrastructure (Operating System, Logging and Monitoring services, Cluster orchestration services, etc.).

PROD environment sizing is provided for 1000 concurrent users with 5 second think time. Where from ACE perspective the user is one API the call to the ACE Dynamic APIs. Please keep in mind that Sizing is provided for ACE overhead only. It can change Depending on complexity and availability of dependant systems, that ACE is accessing.

Sizing units are explained in this document. Total service requirements can be obtained by multiplying number of pod replicas with resources required.

In Sizing DEV/TEST and PROD means the environment type. If it is necessary to deploy both DEV and TEST environments, then required resources are 2 times the sizing provided below.

Service	DEV/TEST (Pods, vCPU, RAM, Disk)	PROD (Pods, vCPU, RAM, Disk)
ace-designer	1, 250m, 256Mi, 5Gb	Not intended for use in production.
ace-runtime-server	1, 500m, 256Mi, 5Gb	2, 500m, 512Mi, 5Gb
ace-deployment-server	1, 250m, 256Mi, 5Gb	1, 500m, 256Mi, 5Gb
bullmq-board	1, 250m, 128Mi, 5Gb	1, 500m, 256Mi, 5Gb

Services

caution

All environment variable keys and values below are case-sensitive

Designer Web

Designer Web application is ACE Designer version for use in browser. It requires Designer runtime server in deployment mode in order to test and execute Dynamic APIs and flows.

Designer Web is the Single Page Application that itself contains only UI code for ACE Designer. It is deployed as the static "Javascript/HTML" package with backend APIs.

Container name

• ace-designer

Deployment options

DEV/TEST	PROD
Required if need to use Designer in browser	Not intended for use in production

Container port

Port exposed by docker image is 5090

Configuration options

See Designer Web configuration for detailed configuration options.

Designer Desktop

Designer Desktop application is ACE Designer version for use on local machine. It is self-sufficient and brings up localhost HTTP server (by default on port 5050) which can be used to test APIs locally.

API Gateway

API gateway which is used to safely (by adding authentication and rate limiting policies for example) expose ACE services to the outside world.

It can also be used to add SSL certificate and expose ACE services via HTTPS protocol if it is not done by load balancer.

API gateway choice is client specific, most common options are cloud based solutions like Azure API Gateway or Amazon API Gateway. Good option for on-premise solution is Kong gateway.

Deployment options

DEV/TEST	PROD
Add additional authentication options to Dynamic APIs, expose ACE services for performance profiling, etc.	Use to expose Dynamic APIs to outside world.

Runtime server

Designer runtime server implements Flow Runner Runtime and hosts Dynamic APIs and Development APIs.

Container name

• ace-runtime-server

Container port

Port exposed by docker image is 5050

Deployment options

DEV/TEST	PROD
Required. Development mode should be enabled if Designer Web is deployed.	Required. Development mode must be disabled.

Configuration options

See Runtime server configuration for detailed configuration options.

Deployment API server

This service provides an API interface for Mongo based workspace deployments and other deployment related actions.

The ACE CLI tool comes pre-packaged with this service and can be used by running the command ace-cli.

The CLI tool is necessary for deploying workspaces and installing schedules in production.

Container name

• ace-deployment-server

Container port

Port exposed by docker image is 5051

Configuration options

See Deployment server configuration for detailed configuration options.

Versioning

legacy

Versioning service is removed from ACE 5.31.0

Versioning is container which ships with ace-versioning CLI. It is used for scheduled job deployment and container is run as one-off script.

Container name

• ace-versioning

Deployment options

DEV/TEST	PROD
Required if scheduled jobs are used	Required if scheduled jobs are used

Configuration options

• REDIS_CACHE_HOST - redis host.
• REDIS_CACHE_PORT - redis port.
• REDIS_CACHE_USER - redis username.
• REDIS_CACHE_KEY - redis authentication key.
• REDIS_KEY_PREFIX - prefix for redis cache keys (default: ace-cache).
• REDIS_CACHE_DB - redis database number
• REDIS_CACHE_TLS_ENABLED - Flag, set to true if redis is tls configured.
• REDIS_CACHE_TLS_KEY - TLS private key
• REDIS_CACHE_TLS_CERT - TLS cert
• REDIS_CACHE_TLS_CA - TLS CA
• REDIS_CACHE_URL - Fully qualified URL with redis:// or secure rediss:// protocol. If provided, *-HOST, *-PORT, *-KEY, *-CERT, *-CA variables will be ignored.
• SCHEDULER_REDIS_HOST - redis host.
• SCHEDULER_REDIS_PORT - redis port.
• SCHEDULER_REDIS_USER - redis username.
• SCHEDULER_REDIS_PASSWORD - redis authentication key.
• SCHEDULER_REDIS_PREFIX - prefix for redis cache keys (default: bull).
• SCHEDULER_REDIS_TLS_ENABLED - Flag, set to true if redis is tls configured.
• SCHEDULER_REDIS_TLS_KEY - TLS private key
• SCHEDULER_REDIS_TLS_CERT - TLS cert
• SCHEDULER_REDIS_TLS_CA - TLS CA
• SCHEDULER_REDIS_URL - Fully qualified URL with redis:// or secure rediss:// protocol. If provided, *-HOST, *-PORT, *-KEY, *-CERT, *-CA variables will be ignored.
• WORKSPACE_MOUNT_PATH _(default "/var/workspace") - path to ace workspace

Sizing

Versioning image is used as one-off script.

Mongo

Mongo can optionally be used to enable persistent user settings. It is also used for dynamic deployment (as opposed to file based Docker image deployment).

Deployment options

DEV/TEST	PROD
Required if using persistent user settings or Dynamic deployment.	Required if using Dynamic deployment.

Scaling

Refer to the Mongo documentation for scaling.

Redis

Redis is used for step level cache and scheduled job persistence. It is recommended to have a separate instance for cache and scheduler as it allows better control over stored data.

caution

Note that scheduler requires redis instance to have noeviction policy to reliably run scheduled jobs. With other policies scheduled job entries can be removed from redis before execution. It might not be the best solution for cache purposes if Redis instance is shared. As Redis doesn't allow eviction policy per DB it's necessary to use a separate instance if different eviction policies are required.

Deployment options

DEV/TEST	PROD
Required if cache or scheduler is used.	Required if cache or scheduler is used.

Scaling

Refer to the Redis documentation about how to scale the Redis.

Keycloak

Keycloak is used for Designer Web authentication. It can optionally be used for Dynamic API protection, but API gateway based approach is recommended.

Deployment options

DEV/TEST	PROD
Required for Designer Web authentication.	Not required unless used for ACE Dynamic API protection.

Scaling

Refer to the Mongo documentation for scaling.

BullMQ Admin

Default BullMQ administration tool (Bull Board) for which is used for scheduled jobs. It allows to view and manage scheduled jobs.

It is not mandatory, and it's possible to use any other BullMQ management tool if required.

If ACE scheduler feature is not used, then this service is not necessary.

Container name

• bullmq-board

Container port

Port exposed by docker image is 3000

Configuration options

• PORT (default "3000") - port used by service
• BULL_PREFIX (default "bull") - BullMQ prefix
• REDIS_HOST - Redis host
• REDIS_PORT - Redis port
• REDIS_PASSWORD - Redis authentication key
• REDIS_DB - Redis DB number
• REDIS_USE_TLS - Flag, set to true if redis is tls configured.
• REDIS_TLS_KEY - TLS private key
• REDIS_TLS_CERT - TLS cert
• REDIS_TLS_CA - TLS CA

Deployment options

DEV/TEST	PROD
Recommended if scheduler is used.	Recommended if scheduler is used.

Designer proxy

legacy

Designer proxy service is removed starting with ACE version 5.28.0. Service can be safely removed when moving to a newer version.

Repositories other than azure devops are not supported in web by isomorphic-git. To support these repositories proxy is required.

Container name

• ace-designer-proxy

Deployment options

DEV/TEST	PROD
required when repositories other then azure devops are used	required when repositories other then azure devops are used

Configuration options

• Generic options
  • LOG_LEVEL (default "info") - service log verbosity level. Possible values error, warn, info, http, verbose, debug, silly.

Scaling

All ACE services can be scaled horizontally. For DB scaling see corresponding DB documentation.

Logging

All ACE services are logging according to env variable LOG_LEVEL supplied to the service. It is possible to capture logs in any Log aggregation solution by capturing what is being logged to stdout.

To prevent GDPR related issues sensitive data is not logged by default

User can provide following env variables in 'Settings' related to logs. FULL_DATA_LOGGING = true to enable logging of sensitive data. ACE_LOG_EXCHANGE = true to log request(url, method, body) and response(statusCode, body) related data at info level.

It is also possible to log custom business information using "Logger" step in flow

Security

When exposing Dynamic APIs to Consumers use API Gateway or any other infrastructure components to secure Dynamic APIs.

Monitoring

There are no considerable warmups livenessProbe and readinessProbe can be defined on per $HOST:$Port basis where /health endpoint is not defined.

Health endpoint /health in services can be used to check health of service. Available in services ace-runtime-server,ace-designer, ace-deployment-server.

Diagnostics endpoint /diagnostics in services can be used to check dependant services and databases connectivity. Available in services ace-runtime-server, ace-designer, ace-deployment-server.