This review focuses on progress and emerging challenges in experimentally validated modeling of microscale urban thermal environments over the last two decades. In the last few decades, there has been a surge in urban energy contribution resulting in elevated urban day-/night-time air temperatures. While there is no single solution to urban heat, mitigation strategies can be implemented to minimize the harmful effects of urban heat both on humans and the environment. To study the effects of urban heat, numerical modeling of urban thermal environments has seen a rise in usage of several application specific atmospheric modeling software packages, and multiple studies and reviews have already covered the prolific engineering use cases. However, there are inherent and unintentional biases introduced by each modeling software package, that inhibit validity and accuracy for general engineering use. This review critically analyzes the limitations of current state-of-the-art (SOA) microscale atmospheric modeling approaches and identify necessary areas for improvement. Urban thermal environment models must be validated with measurements to gain confidence in the predictive capabilities. This review will additionally examine the next generation of measurement techniques that leverage advances in computing and communications to create distributed meteorological sensor networks for improved spatial and temporal resolutions, that can provide a rich platform for model validation. High fidelity and accurate simulations of urban thermal environments improve confidence in the study of urban heat, its mitigation, and its impact on urban engineering applications in building energy usage and sustainability.