CARLA Simulator on WSL2: Resolving the Docker Vulkan Trap

1. System & Hardware Specifications

The configuration and environment details used:

• Host OS: Windows 11
• WSL2 Distro: Ubuntu 24.04 LTS (Kernel: 6.6.87.2-microsoft-standard-WSL2)
• GPU: NVIDIA GeForce RTX 5060 Laptop (8GB VRAM, Host Driver: 592.27, CUDA 13.1)
• Docker: Docker Desktop 4.63.0 (Engine 29.2.1)
• CARLA Version: 0.9.16

2. The Working Solution: Windows Server + WSL2 Client Hybrid Architecture

Instead of fighting virtualization layers and containerized Vulkan driver mapping, the most performant and stable approach for CARLA on WSL2 is a hybrid architecture:

1. CARLA Server runs natively on Windows: Gives Unreal Engine 4 direct, high-performance access to the host GPU (RTX 5060) for 3D rendering and physics.
2. CARLA Client runs inside WSL2: Keeps the Python development, training pipelines, and model libraries in the native Linux environment.

┌─────────────────────────────────────────────┐
│  Windows Host (Direct GPU Access)           │
│  CARLA Simulator Server (CarlaUE4.exe)      │
│  Listens on Port 2000                       │
│  Handles: 3D Rendering, Physics, Autopilot  │
└──────────────────┬──────────────────────────┘
                   │
                   │ Network (127.0.0.1:2000)
                   │
┌──────────────────▼──────────────────────────┐
│  WSL2 (Ubuntu 24.04 Dev Environment)         │
│  Python Client (carenv Conda Environment)    │
│  Handles: Agent Logic, RL training, Sensors │
└─────────────────────────────────────────────┘

Installation Steps

Step 1: Windows Side Setup

1. Download CARLA_0.9.16Windows.zip from the CARLA GitHub Releases.
2. Extract the zip file to your preferred directory (referred to as <CARLA_PATH> here, e.g., C:\CARLA_0.9.16).
3. Install the required dependencies:
   • DirectX End-User Runtime: Download and run dxwebsetup.exe.
   • Visual C++ Redistributable 2015-2022 (x64): Download and run VC_redist.x64.exe.

Step 2: WSL2 Side Setup

Create and activate a conda environment for the CARLA client:

conda create -n carenv python=3.10 -y
conda activate carenv
pip install carla

Verify the installation:

python -c "import carla; print('CARLA client package successfully installed!')"

Step 3: Network Configuration (Mirrored Mode)

To ensure seamless network communication between WSL2 and Windows via 127.0.0.1, enable Mirrored mode in WSL2.

Create or edit your Windows user profile configuration file %USERPROFILE%\.wslconfig:

[wsl2]
networkingMode=mirrored

Then, restart WSL2 from a Windows PowerShell terminal to apply the changes:

wsl --shutdown

3. Running the Simulation

Step 1: Start the CARLA Server (Windows)

Open a Windows PowerShell terminal and start the server:

cd <CARLA_PATH>

# Options:
# 1. With visual window (default)
.\CarlaUE4.exe

# 2. Headless mode (no visual output, lower VRAM overhead)
.\CarlaUE4.exe -RenderOffScreen -nosound

# 3. Custom window resolution
.\CarlaUE4.exe -windowed -ResX=1280 -ResY=720

Step 2: Run Python Client Scripts (WSL2)

Once the server is running on the Windows side, launch the client scripts inside WSL2:

1. Connection Test (test_connection.py)

"""CARLA 连接测试脚本 - 从 WSL2 连接 Windows CARLA 服务端"""
import carla

client = carla.Client('127.0.0.1', 2000)
client.set_timeout(10.0)

print(f"Server version: {client.get_server_version()}")
print(f"Client version: {client.get_client_version()}")
print(f"可用地图: {client.get_available_maps()}")

world = client.get_world()
print(f"当前地图: {world.get_map().name}")
print(f"当前 actors 数量: {len(world.get_actors())}")
print("\n连接测试通过!")

2. Spawn and Control Vehicle (spawn_vehicle.py)

This script spawns a Tesla Model 3, attaches a front-facing camera, enables autopilot, and saves camera frames to a local folder for 10 seconds:

"""CARLA 生成车辆 + 摄像头截图"""
import carla
import random
import time
import os

OUTPUT_DIR = "output"
os.makedirs(OUTPUT_DIR, exist_ok=True)

# 连接
client = carla.Client('127.0.0.1', 2000)
client.set_timeout(10.0)
world = client.get_world()

# 获取蓝图库
blueprint_library = world.get_blueprint_library()

# 找一个车辆蓝图
vehicle_bp = blueprint_library.find('vehicle.tesla.model3')
spawn_points = world.get_map().get_spawn_points()
spawn_point = random.choice(spawn_points)

# 生成车辆
vehicle = world.spawn_actor(vehicle_bp, spawn_point)
print(f"车辆已生成: {vehicle.type_id} at {spawn_point.location}")

# 添加摄像头传感器
camera_bp = blueprint_library.find('sensor.camera.rgb')
camera_bp.set_attribute('image_size_x', '800')
camera_bp.set_attribute('image_size_y', '600')
camera_bp.set_attribute('fov', '90')

# 摄像头放在车顶上方
camera_transform = carla.Transform(carla.Location(x=1.5, z=2.4))
camera = world.spawn_actor(camera_bp, camera_transform, attach_to=vehicle)
print(f"摄像头已挂载")

# 保存截图计数
frame_count = [0]

def save_image(image):
    frame_count[0] += 1
    filepath = os.path.join(OUTPUT_DIR, f"frame_{frame_count[0]:04d}.png")
    image.save_to_disk(filepath)
    print(f"截图保存: {filepath}")

camera.listen(save_image)

# 让车辆自动驾驶
vehicle.set_autopilot(True)
print("自动驾驶已开启")

# 运行 10 秒，采集截图
print("开始采集，运行 10 秒...")
time.sleep(10)

# 清理
camera.stop()
camera.destroy()
vehicle.destroy()
print(f"完成! 共采集 {frame_count[0]} 帧截图，保存在 {OUTPUT_DIR}")

4. The Failed Path: Running CARLA inside WSL2 Docker (Troubleshooting Record)

For completeness, below is a record of our attempts to run CARLA directly inside a WSL2 Docker container, and why it consistently failed.

Attempted Run Configurations

Attempt 1: Basic GPU-enabled Container

docker run -d \
    --name carla \
    --gpus all \
    --net=host \
    -e NVIDIA_VISIBLE_DEVICES=all \
    -e NVIDIA_DRIVER_CAPABILITIES=all \
    carlasim/carla:0.9.16 \
    bash CarlaUE4.sh -RenderOffScreen -nosound

• Result: The engine complains: WARNING: lavapipe is not a conformant vulkan implementation. After 60 seconds, it crashes with GameThread timed out waiting for RenderThread.

Attempt 2: Mount Host Vulkan ICD File

docker run -d \
    --name carla \
    --gpus all \
    --net=host \
    -e NVIDIA_VISIBLE_DEVICES=all \
    -e NVIDIA_DRIVER_CAPABILITIES=all \
    -e VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/nvidia_icd.json \
    -v /usr/share/vulkan/icd.d:/usr/share/vulkan/icd.d:ro \
    carlasim/carla:0.9.16 \
    bash CarlaUE4.sh -RenderOffScreen -nosound

• Result: Immediate Segmentation Fault.

Attempt 3: Using the Official --runtime=nvidia Flag

docker run -d \
    --name carla \
    --runtime=nvidia \
    --net=host \
    -e NVIDIA_VISIBLE_DEVICES=all \
    -e NVIDIA_DRIVER_CAPABILITIES=all \
    carlasim/carla:0.9.16 \
    bash CarlaUE4.sh -RenderOffScreen -nosound

• Result: The same lavapipe warning, followed by a render thread timeout crash after 60 seconds.

Attempt 4: Forcing OpenGL Mode

docker run -d \
    --name carla \
    --gpus all \
    --net=host \
    -e NVIDIA_VISIBLE_DEVICES=all \
    -e NVIDIA_DRIVER_CAPABILITIES=all \
    -e DISPLAY=:0 \
    carlasim/carla:0.9.16 \
    bash CarlaUE4.sh -RenderOffScreen -nosound -opengl

• Result: Crash. WSL2 does not expose a /dev/dri/ device inside the container, making OpenGL hardware acceleration unavailable.

Root Cause Analysis

The rendering failure in the container is caused by three layered issues:

1. The nvidia_icd.json Directory Bug

In Ubuntu 24.04:

$ file /usr/share/vulkan/icd.d/nvidia_icd.json
/usr/share/vulkan/icd.d/nvidia_icd.json: directory

A post-installation package manager bug mistakenly runs mkdir -p on the path, converting the configuration JSON file into an empty folder. This breaks standard Vulkan loader lookups.

• Fix: sudo rm -rf /usr/share/vulkan/icd.d/nvidia_icd.json

2. Missing Vulkan Mapping Libraries in WSL2

Even with the JSON path fixed, WSL2 is missing NVIDIA’s Vulkan mapping libraries. Under /usr/lib/wsl/lib/ (and its Windows host counterpart C:\Windows\System32\lxss\lib\), we can see:

• libcuda.so (CUDA computation works)
• libd3d12.so (DirectX 12 works)
• libnvidia-ml.so.1 (nvidia-smi works)
• libvulkan_nvidia.so (❌ Missing!)
• libGLX_nvidia.so (❌ Missing!)

NVIDIA’s host drivers do not map the Vulkan/OpenGL shared libraries into the WSL2 environment for Blackwell-based Laptop GPUs (like the RTX 5060) on host driver version 592.27.

3. Software Renderer Fallback

Because no hardware Vulkan/OpenGL drivers are visible to WSL2, the Vulkan loader falls back to llvmpipe / lavapipe (CPU-based software rendering):

$ VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/gfxstream_vk_icd.x86_64.json vulkaninfo --summary
ERROR: setup_loader_term_phys_devs: Failed to detect any valid GPUs in the current config

CARLA (based on Unreal Engine 4.26) requires hardware rasterization. Forcing the simulation to render on the CPU leads to massive lag, eventually triggering the GameThread timed out waiting for RenderThread watchdog mechanism.

5. Summary of Tips & Troubleshooting

• VRAM Optimization: The RTX 5060 Laptop has 8GB VRAM. Running CARLA with a visual window in standard mode occupies around 7.7GB of VRAM. If you run out of memory (OOM), run the server with -quality-level=Low or run it offscreen (-RenderOffScreen).
• Networking Issues: If the client fails to connect to 127.0.0.1, check your Windows Firewall settings to ensure port 2000 is open, and verify that networkingMode=mirrored is active in WSL2.
• Docker Cleanup: Since Dockerized CARLA is unusable in this setup, free up disk space by removing the unused container image:

  docker rmi carlasim/carla:0.9.16
  

6. Useful Resources

• CARLA Official Documentation
• CARLA Issue #5806 (WSL2 Discussion)
• NVIDIA CUDA on WSL Guide