Automating Visual Tasks on Windows 11 with Python RPA

This guide gives the process of setting up and using Python RPA for automating visual tasks on Windows 11. Python RPA, powered by the TagUI framework, provides a robust and flexible solution for automating various desktop applications and web interactions.

Prerequisites

• Python 3.12 or above
• OpenJDK
• FFmpeg

Installation

1. Install Python 3.12 and the rpa package:

   choco install python312
   pip install rpa
   

2. Set up TagUI:

   • Download TagUI from the official releases page: https://github.com/kelaberetiv/TagUI/releases/download/v6.110.0/TagUI_Windows.zip
   • Extract the downloaded ZIP file to C:\.
   • Install OpenJDK from the official website: https://corretto.aws/downloads/latest/amazon-corretto-8-x64-windows-jdk.msi
   • Update the system environment variables to include the c:\tagui\src directory. Refer to this guide for instructions: https://www.c-sharpcorner.com/article/add-a-directory-to-path-environment-variable-in-windows-10/

   Note: If you encounter the MSVCR110.dll is missing issue, install the vcredist_x86.exe package.

3. Install FFmpeg:

   • Download the full FFmpeg build from the official website: https://www.gyan.dev/ffmpeg/builds/ffmpeg-release-full.7z
   • Extract the downloaded archive and copy the ffmpeg.exe file from the bin directory to the base directory of the Python RPA project.MSVCR110.dll is missing issue

CPU Usage Comparison: OBS vs. FFmpeg

When comparing the CPU usage between OBS and FFmpeg for screen recording on Windows 11, FFmpeg demonstrates significantly lower CPU utilization, especially at higher frame rates. The table below shows the CPU usage range for an 8-core system:

Software	FPS	CPU Usage Range (8 Cores)
OBS (1 FPS)	1	60-65%
OBS (30 FPS)	30	>90%
FFmpeg GDI Screengrabber (1 FPS)	1	5-8%
FFmpeg GDI Screengrabber (30 FPS)	30	30-35%

It’s important to note that DirectShow is not recommended for virtual machines as it requires a physical display for screen recording or RDP.