Deep-RL Target Wake Time Scheduling for IEEE 802.11ax
Wi-Fi networks serving heterogeneous IoT and multimedia devices suffer from CSMA contention, resulting in performance anomaly and excessive energy use. Target Wake Time (TWT) improves this by slicing the beacon interval into non-overlapping service periods (SPs) and assigning one or more stations to each SP. However, optimal 802.11ax TWT scheduling is NP-hard (≈1030 configurations for 32 stations). To solve this, I developed an end-to-end RL scheduler in an ns-3 simulator with a custom BSR Manager, Wi-Fi-stack patches for unilateral TWT, a C++↔Python bridge over ns3-ai shared memory, and PPO agents (MLP and LSTM) trained on a 208-dimensional protocol-compliant observation space with a 380-configuration action space and three reward presets. Against an analytical M/D/1 baseline: −44% energy, +18% throughput, −73% packet drops (MLP-PPO, energy preset) and −77% drops (LSTM-PPO, queue preset). Because observations are restricted to standard BSR and 802.11k statistics, the scheduler is deployable on existing APs with no firmware changes.
Stack: C++ (ns-3 3.44), Python, Stable-Baselines3 PPO, ns3-ai shared memory, one-shot build→train→evaluate pipeline.
