With the advance in sensing, actuation, and mechanical design, the robots are becoming more capable of performing complex and high-precision tasks ranging from autonomous driving in urban cities to handling packages in fulfillment centers. Control and planning of robot motion in real-world environments require high-fidelity models to simulate how the robot's motion impacts physical objects before they take action. Many well-established approaches in robotics rely on mathematical models due to their analytic tractability; however as robotic systems become complex and are required to interact in physical environments, we face limitations in finding analytically tractable models and begin to resort to new models that leverage the powerful computational resources embodied in robotic systems. To address the challenges in robot operation in physical environments, in this project, we explore the idea of building computational models based on physics engines and design feedback algorithms to autotune parameters of the model whenever the robot detects the discrepancy between the simulation and its experience in the physical world -- to reduce the sim-to-real gap. As key applications, we imagine to apply outcomes of the project to enable robotic manipulators to carry out the tasks in assembly lines and research laboratories that are labor-intensive and require dexterous skills.