Addition

$$ \begin{aligned} & c = a + b \\ \\ & \frac{dc}{da} = \frac{((a + h) + b) - (a + b)}{h} \\ \\ & \frac{dc}{da} = \frac{(a + h + b) - (a + b)}{h} \\ \\ & \frac{dc}{da} = \frac{a + h + b - (a + b)}{h} \\ \\ & \frac{dc}{da} = \frac{a + h + b - a - b}{h} \\ \\ & \frac{dc}{da} = \frac{a - a + h + b - b}{h} \\ \\ & \frac{dc}{da} = \frac{h}{h} \\ \\ & \frac{dc}{da} = 1 \\ \end{aligned} $$

Multiplication

$$ \begin{aligned} & c = a * b \\ \\ & \frac{dc}{da} = \frac{((a + h) * b) - (a * b)}{h} \\ \\ & \frac{dc}{da} = \frac{((a * b) + (h * b)) - (a * b)}{h} \\ \\ & \frac{dc}{da} = \frac{(a * b) + (h * b) - (a * b)}{h} \\ \\ & \frac{dc}{da} = \frac{(a * b) - (a * b) + (h * b)}{h} \\ \\ & \frac{dc}{da} = \frac{(h * b)}{h} \\ \\ & \frac{dc}{da} = \frac{h * b}{h} \\ \\ & \frac{dc}{da} = \frac{h}{h} * \frac{b}{1} \\ \\ & \frac{dc}{da} = 1 * \frac{b}{1} \\ \\ & \frac{dc}{da} = \frac{b}{1} \\ \\ & \frac{dc}{da} = b \\ \end{aligned} $$

Matrices

Links

Vector, Matrix, and Tensor Derivatives by Erik Learned-Miller