S_{i} = \frac{e^{a_{i}}}{\sum_{k=1}^{n}e^{a_{k}}} = \frac{e^{a_{i}}}{(e^{a_{1}} + e^{a_{2}} + e^{a_{3}})} \\ \frac{\partial S_{i}}{\partial a_{j}} = \frac{\partial ...
calculate partial derivatives instead of total derivative like tf.gradients with parameter stop_gradients https://www.tensorflow.org/api_docs/python/tf/gradients It ...