neural_lam.utils.tensor
=======================

.. py:module:: neural_lam.utils.tensor

.. autoapi-nested-parse::

   Numerically stable inverses of common activation functions.



Functions
---------

.. autoapisummary::

   neural_lam.utils.tensor.inverse_sigmoid
   neural_lam.utils.tensor.inverse_softplus


Module Contents
---------------

.. py:function:: inverse_sigmoid(x: torch.Tensor) -> torch.Tensor

   Inverse of ``torch.sigmoid`` with clamping for numerical stability.

   Sigmoid output takes values in ``[0, 1]``; we clamp the input slightly
   within that open interval before applying ``log(x / (1 - x))``.

   Note that ``torch.clamp`` will make gradients 0 near the bounds, but
   this is not a problem as values of x that are this close to 0 or 1
   have gradients of 0 anyhow.

   :param x: Input tensor assumed to contain logits after a sigmoid.
   :type x: torch.Tensor

   :returns: Tensor containing ``log(x / (1 - x))`` after clamping away from the
             saturation limits.
   :rtype: torch.Tensor

   .. rubric:: Notes

   ``torch.clamp`` zeroes gradients for values at the bounds, but values this
   close to 0 or 1 already have negligible gradients.


.. py:function:: inverse_softplus(x: torch.Tensor, beta: float = 1.0, threshold: float = 20.0) -> torch.Tensor

   Inverse of :func:`torch.nn.functional.softplus`.

   For most inputs this function is exact up to numerical precision. The
   input is clamped to ensure numerical stability: values above
   ``threshold / beta`` are treated as linear (which is exact in that
   regime), and values near zero are clamped to avoid ``log`` of
   non-positive numbers. Only near the lower clamping bound does the
   result deviate from the true inverse.

   :param x: Input tensor whose softplus inverse should be computed.
   :type x: torch.Tensor
   :param beta: Softplus ``beta`` parameter that controls the sharpness. Default ``1``.
   :type beta: float, optional
   :param threshold: Threshold above which the function is treated as linear for numerical
                     stability. Default ``20``.
   :type threshold: float, optional

   :returns: Tensor containing the inverse-softplus values.
   :rtype: torch.Tensor

   .. rubric:: Notes

   ``torch.clamp`` will zero the gradients near the bounds, but values this
   close to zero or ``threshold / beta`` already have negligible gradients.