With 2000km of coral reef and marine life surrounding it, the Red Sea is a rich and diverse ecosystem. Due to its ecological and economic importance, persistent monitoring of its key areas is critical for marine biology research, economic development, and environmental protection. However, long-term survey and observation of extensive marine environments are extremely labor-intensive and hence require new technological development. There have been a numerous efforts made in research and development of robots designed to autonomously survey and collect data in remote areas that are hard to access by human divers or require long-term and repeated monitoring. However, there remain technical challenges in implementing long-term autonomy in underwater robotics to allow a robot to navigate in close proximity of marine lives over an extended period of time while ensuring absolute safety and not interfering with their natural habitats. In this project, we explore learning-based planning/control approaches that combine optimization-based design and data-driven machine learning methods. Leveraging the optimal feedback control and reinforcement learning, we aim to develop a framework that enables the robot to 1) learn how the environmental characteristics (light condition, current speed, etc.) affect its navigation performance and 2) adaptively revise its control policies to safely and graciously navigate around marine habitats.