nonlinearities¶
This module contains a collection of physical and aphysical activation functions. Nonlinearities can be incorporated into an optical neural network by using the Activation(nonlinearity) NetworkLayer.
-
class
neuroptica.nonlinearities.Abs(N, mode='polar')[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityRepresents a transformation z -> |z|. This can be called in any of “full”, “condensed”, and “polar” modes
-
__init__(N, mode='polar')[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
dIm_dIm(a: <MagicMock id='139905357337880'>, b: <MagicMock id='139905357350392'>)[source]¶ Gives the derivative of the imaginary part of the nonlienarity w.r.t. the imaginary part of the input
-
dIm_dRe(a: <MagicMock id='139905357308760'>, b: <MagicMock id='139905357325368'>)[source]¶ Gives the derivative of the imaginary part of the nonlienarity w.r.t. the real part of the input
-
dRe_dIm(a: <MagicMock id='139905358328216'>, b: <MagicMock id='139905357296248'>)[source]¶ Gives the derivative of the real part of the nonlienarity w.r.t. the imaginary part of the input
-
dRe_dRe(a: <MagicMock id='139905358294936'>, b: <MagicMock id='139905358315704'>)[source]¶ Gives the derivative of the real part of the nonlienarity w.r.t. the real part of the input
-
df_dIm(a: <MagicMock id='139905357396120'>, b: <MagicMock id='139905357408632'>)[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-
df_dRe(a: <MagicMock id='139905357362904'>, b: <MagicMock id='139905357379512'>)[source]¶ Gives the derivative of the nonlinearity with respect to the real part alpha of the input
-
df_dphi(r: <MagicMock id='139905357446168'>, phi: <MagicMock id='139905357462776'>)[source]¶ Gives the derivative of the nonlinearity with respect to the angle phi of the input
-
-
class
neuroptica.nonlinearities.AbsSquared(N)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityMaps z -> |z|^2, corresponding to power measurement by a photodetector.
-
__init__(N)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dphi(r: <MagicMock id='139905357525288'>, phi: <MagicMock id='139905357541896'>)[source]¶ Gives the derivative of the nonlinearity with respect to the angle phi of the input
-
-
class
neuroptica.nonlinearities.ComplexNonlinearity(N, holomorphic=False, mode='condensed')[source]¶ Bases:
neuroptica.nonlinearities.NonlinearityBase class for a complex-valued nonlinearity
-
__init__(N, holomorphic=False, mode='condensed')[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
backward_pass(gamma: <MagicMock id='139905357886576'>, Z: <MagicMock id='139905357903184'>) → <MagicMock id='139905357911600'>[source]¶ Backpropagate a signal through the layer :param gamma: backpropagated signal from the (l+1)th layer :param Z: output fields from the forward_pass() run :return: backpropagated fields delta_l
-
dIm_dIm(a: <MagicMock id='139905358702128'>, b: <MagicMock id='139905358714640'>) → <MagicMock id='139905358727152'>[source]¶ Gives the derivative of the imaginary part of the nonlienarity w.r.t. the imaginary part of the input
-
dIm_dRe(a: <MagicMock id='139905358660496'>, b: <MagicMock id='139905358673008'>) → <MagicMock id='139905358685520'>[source]¶ Gives the derivative of the imaginary part of the nonlienarity w.r.t. the real part of the input
-
dRe_dIm(a: <MagicMock id='139905358618864'>, b: <MagicMock id='139905358631376'>) → <MagicMock id='139905358643888'>[source]¶ Gives the derivative of the real part of the nonlienarity w.r.t. the imaginary part of the input
-
dRe_dRe(a: <MagicMock id='139905358040592'>, b: <MagicMock id='139905358057200'>) → <MagicMock id='139905358069712'>[source]¶ Gives the derivative of the real part of the nonlienarity w.r.t. the real part of the input
-
df_dIm(a: <MagicMock id='139905358007152'>, b: <MagicMock id='139905358019664'>) → <MagicMock id='139905358032176'>[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-
df_dRe(a: <MagicMock id='139905357961424'>, b: <MagicMock id='139905357973936'>) → <MagicMock id='139905357990544'>[source]¶ Gives the derivative of the nonlinearity with respect to the real part alpha of the input
-
df_dZ(Z: <MagicMock id='139905357936400'>) → <MagicMock id='139905357948912'>[source]¶ Gives the total complex derivative of the (holomorphic) nonlinearity with respect to the input
-
df_dphi(r: <MagicMock id='139905358781296'>, phi: <MagicMock id='139905358797904'>) → <MagicMock id='139905358810416'>[source]¶ Gives the derivative of the nonlinearity with respect to the angle phi of the input
-
-
class
neuroptica.nonlinearities.ElectroOpticActivation(N, alpha=0.1, responsivity=0.8, area=1.0, V_pi=10.0, V_bias=10.0, R=1000.0, impedance=<MagicMock name='mock.__rmul__()' id='139905358169928'>, g=None, phi_b=None)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityElectro-optic activation function with intensity modulation (remod).
This activation can be configured either in terms of its physical parameters, detailed below, or directly in terms of the feedforward phase gain, g and the biasing phase, phi_b.
If the electro-optic parameters below are specified g and phi_b are computed for the user.
alpha: Amount of power tapped off to PD [unitless] responsivity: PD responsivity [Watts/amp] area: Modal area [micron^2] V_pi: Modulator V_pi (voltage required for a pi phase shift) [Volts] V_bias: Modulator static bias [Volts] R: Transimpedance gain [Ohms] impedance: Characteristic impedance for computing optical power [Ohms]-
__init__(N, alpha=0.1, responsivity=0.8, area=1.0, V_pi=10.0, V_bias=10.0, R=1000.0, impedance=<MagicMock name='mock.__rmul__()' id='139905358169928'>, g=None, phi_b=None)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dIm(a: <MagicMock id='139905358236584'>, b: <MagicMock id='139905358253192'>) → <MagicMock id='139905358265704'>[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-
-
class
neuroptica.nonlinearities.LinearMask(N: int, mask=None)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityTechnically not a nonlinearity: apply a linear gain/loss to each element
-
__init__(N: int, mask=None)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
-
class
neuroptica.nonlinearities.SPMActivation(N, gain)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityLossless SPM activation function
phase_gain [ rad/(V^2/m^2) ] : The amount of phase shift per unit input “power”-
__init__(N, gain)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dIm(a: <MagicMock id='139905358099064'>, b: <MagicMock id='139905358115672'>) → <MagicMock id='139905358124088'>[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-
-
class
neuroptica.nonlinearities.Sigmoid(N)[source]¶ Bases:
neuroptica.nonlinearities.NonlinearitySigmoid activation; maps z -> 1 / (1 + np.exp(-z))
-
class
neuroptica.nonlinearities.SoftMax(N)[source]¶ Bases:
neuroptica.nonlinearities.NonlinearityApplies softmax to the inputs. Do not use in with categorical cross entropy, which implicitly includes this.
-
class
neuroptica.nonlinearities.bpReLU(N, cutoff=1, alpha=0)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityDiscontinuous (but holomorphic and backpropable) ReLU f(x_i) = alpha * x_i if |x_i| < cutoff f(x_i) = x_i if |x_i| >= cutoff
cutoff: value of input |x_i| above which to fully transmit, below which to attentuate alpha: attenuation factor f(x_i) = f-
__init__(N, cutoff=1, alpha=0)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
-
class
neuroptica.nonlinearities.cReLU(N)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityContintous, but non-holomorphic and non-simply backpropabable ReLU of the form f(z) = ReLU(Re{z}) + 1j * ReLU(Im{z}) see: https://arxiv.org/pdf/1705.09792.pdf
-
__init__(N)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dIm(a: <MagicMock id='139905356797488'>, b: <MagicMock id='139905356805904'>) → <MagicMock id='139905356818416'>[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-
-
class
neuroptica.nonlinearities.modReLU(N, cutoff=1)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityContintous, but non-holomorphic and non-simply backpropabable ReLU of the form f(z) = (|z| - cutoff) * z / |z| if |z| >= cutoff (else 0) see: https://arxiv.org/pdf/1705.09792.pdf (note, cutoff subtracted in this definition)
cutoff: value of input |x_i| above which to-
__init__(N, cutoff=1)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dphi(r: <MagicMock id='139905357226112'>, phi: <MagicMock id='139905357238624'>)[source]¶ Gives the derivative of the nonlinearity with respect to the angle phi of the input
-
-
class
neuroptica.nonlinearities.zReLU(N)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityContintous, but non-holomorphic and non-simply backpropabable ReLU of the form f(z) = z if Re{z} > 0 and Im{z} > 0, else 0 see: https://arxiv.org/pdf/1705.09792.pdf
-
__init__(N)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dIm(a: <MagicMock id='139905356885184'>, b: <MagicMock id='139905356901792'>) → <MagicMock id='139905356914304'>[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-