Post

Prefect Test Env

Prefect Test Env

Prefect Setup

Prefect Server Docker Deployment

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
name: prefect

services:
  prefect:
    image: rsubr/quark:25.05
    container_name: prefect.localhost
    restart: always
		
    command: "prefect server start"
		
    ports:
      - 4200:4200
			
    environment:
      - PREFECT_SERVER_API_HOST=0.0.0.0
      - PREFECT_SERVER_API_AUTH_STRING=admin:password
			
      - PREFECT_API_URL=http://localhost:4200/api
      - PREFECT_API_AUTH_STRING=admin:password
			
      - PREFECT_SERVER_DATABASE_CONNECTION_URL=postgresql+asyncpg://prefect:prefect@localhost:5432/prefect

Work Pools

Create work pools in the UI: http://localhost:4200/

Prefect Worker Deployment

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
name: prefect-worker

services:
  prefect-worker:
    image: rsubr/quark:25.05
    container_name: prefect-worker.localhost
    restart: always
		
    environment:
      - PREFECT_API_URL=http://localhost:4200/api
      - PREFECT_API_AUTH_STRING=admin:password
			
    # Set concurrency limit "-l" to 4 workers
    # Attach to work pool "test"
    command: prefect worker start --with-healthcheck -t process -l 4 -n "worker-1" -p "test"
		
    volumes:
      - /app:/app

Create Deployment in Prefect

  1. Add the following Prefect environment variables to the local machine:

    1
    2
    
     prefect config set PREFECT_API_URL=http://localhost:4200/api
     prefect config set PREFECT_API_AUTH_STRING=admin:password
    
  2. Python code example (/app/prefect-code/sample.py):

    1
    2
    3
    4
    5
    6
    7
    8
    
     from prefect import flow
    
     @flow(
         name="Test",
         log_prints=True,
     )
     def sample():
         print("Hello World!")
    
  3. Create a deployment for this Python code:

    • Run the following command from the /app directory:
      1
      
      prefect deploy
      
    • Choose the flow “Test” from the list.
    • Enter a deployment name.
    • No schedules needed.
    • Select the work pool from the list.
    • No need to pull code from a remote location.
This post is licensed under CC BY 4.0 by the author.