As data collection grows, so does the need for techniques that leverage data while respecting known structure, yet data assimilation and inverse problems remain challenging for numerical methods. Motivated by the increasing relevance of deep learning across scientific disciplines, this thesis investigates the interplay between neural network architecture, loss landscape geometry, and optimization in scientific machine learning. We analyze optimization of physics-informed neural networks (PINNs), which can exhibit ill-conditioned loss landscapes. We develop a randomized sketching algorithm to approximate the natural gradient direction. The spectral structure of the underlying Gram matrix guarantees near-optimal approximation error at a fraction of the computational and memory cost, enabling efficient large-scale training with state-of-the-art accuracy. In more general settings, working in function space, we show that the generalized Gauss-Newton update corresponds to a weighted projection of the Newton direction onto the model's tangent space, while a loss-independent variant projects the gradient itself. This provides insight into why Gauss-Newton-type methods can outperform Newton's method. We study this empirically via the sketching technique developed for PINNs on regression tasks and MNIST. We introduce a structure-preserving neural ODE architecture for PINNs, where collocation points enter as initial conditions and are transformed under a learned flow to approximate PDE solution surfaces, with forward sensitivity equations providing memory-efficient residual derivatives. Finally, we build a convolutional neural network surrogate for the pseudo two-dimensional electrochemical model of lithium-ion batteries. Treating concentration profiles as images parameterized by spatial position and particle radius, the architecture predicts voltage, concentration dynamics, and failure under high-current cycles, and extends to degradation identification via state-of-health parameters.
Maricela Best McKay (Fri,) studied this question.