Algorithmic differentiation allows the approximation of certain non-smooth functions by piecewise affine models. In contrast to linear models, these models are usually more accurate approximations since they reflect underlying non-smoothness. However, the algebraic representation of the piecewise affine models can be come large and, thus, computationally inefficient. As a solution, we propose to approximate the piecewise affine approximation by local models that have a smaller algebraic representation and that are computationally more tractable. The proposed approach is based on combining techniques from interval arithmetic and numerical linear algebra. In particular, we propose a generalized representation for piecewise affine models and their local models. We derive explicit matrix updates enabling to compute local models efficiently. Numerical experiments are reported demonstrating the feasibility of the novel approach.