A number of physical systems can be described by ordinary differential equations. When physics is well understood, the time dependent responses are easily obtained numerically. The particular numerical method used for integration depends on the application. Unfortunately, when physics is not fully understood, the discrepancies between predictions and observed responses can be large and unacceptable. In this paper, we propose an approach that uses observed data to estimate the missing physics in the original model (i.e., model-form uncertainty). In our approach, we first design recurrent neural networks to perform numerical integration of the ordinary differential equations. Then, we implement the recurrent neural network as a directed graph. This way, the nodes in the graph represent the physics-informed kernels found in the ordinary differential equations. We quantify the missing physics by carefully introducing data-driven in the directed graph. This allows us to estimate the missing physics (discrepancy term) even for hidden nodes of the graph. We studied the performance of our proposed approach with the aid of three case studies (fatigue crack growth, corrosion-fatigue crack growth, and bearing fatigue) and state-of-the-art machine learning software packages. Our results demonstrate the ability to perform estimation of discrepancy, reducing gap between predictions and observations, at reasonable computational cost.