Continuous action reinforcement learning for topology optimization via capped-simplex projection

초록

This paper presents a continuous-action reinforcement learning (RL) framework for topology optimization that addresses two inadequately resolved fundamental challenges in existing RL-based approaches: exact satisfaction of the volume constraint and the elimination of geometrically disconnected floating material regions. Unlike existing RL-based topology optimization methods that are restricted to discrete action domains, the proposed framework employs the deep deterministic policy gradient (DDPG) algorithm to enable operations within a continuous action space. This key departure from discrete to continuous density variables naturally admits intermediate density values, enabling smooth, gradual density updates that inherently suppress floating materials and stabilize the learning process. A major contribution of this study is the new application of the capped-simplex projection for the RL-based topological optimization context using a continuous action space. It maps the unbounded actor output onto a physically admissible density field and exactly satisfies the volume constraint at every algorithmic step without penalty terms or post-processing corrections. Furthermore, because the framework operates with continuous density variables, well-established filtering techniques from density-based topology optimization can be directly incorporated. The proposed framework is validated on the MBB, L-shaped, and cantilever beams, demonstrating that the DDPG agent consistently evolves near-optimal, structurally sound topologies through trial-and-error interaction with a finite element analysis environment, without relying on pre-computed training datasets. A single fixed set of hyperparameters applied uniformly across all problems and volume fractions performed consistently across the benchmark problems and volume fractions examined in this study. The proposed framework is intended as a methodological development within RL-based topology optimization, focusing on continuous action representation, structural feasibility, and exact constraint enforcement, rather than replacing or outperforming conventional sensitivity-based topology optimization for standard compliance problems.

키워드