nonlinearities¶
This module contains a collection of physical and aphysical activation functions. Nonlinearities can be incorporated into an optical neural network by using the Activation(nonlinearity) NetworkLayer.
-
class
neuroptica.nonlinearities.Abs(N, mode='polar')[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityRepresents a transformation z -> |z|. This can be called in any of “full”, “condensed”, and “polar” modes
-
__init__(N, mode='polar')[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
dIm_dIm(a: <MagicMock id='140414710928776'>, b: <MagicMock id='140414710941288'>)[source]¶ Gives the derivative of the imaginary part of the nonlienarity w.r.t. the imaginary part of the input
-
dIm_dRe(a: <MagicMock id='140414711952328'>, b: <MagicMock id='140414711964840'>)[source]¶ Gives the derivative of the imaginary part of the nonlienarity w.r.t. the real part of the input
-
dRe_dIm(a: <MagicMock id='140414711927304'>, b: <MagicMock id='140414711939816'>)[source]¶ Gives the derivative of the real part of the nonlienarity w.r.t. the imaginary part of the input
-
dRe_dRe(a: <MagicMock id='140414711902280'>, b: <MagicMock id='140414711910696'>)[source]¶ Gives the derivative of the real part of the nonlienarity w.r.t. the real part of the input
-
df_dIm(a: <MagicMock id='140414710987016'>, b: <MagicMock id='140414710999528'>)[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-
df_dRe(a: <MagicMock id='140414710957896'>, b: <MagicMock id='140414710970408'>)[source]¶ Gives the derivative of the nonlinearity with respect to the real part alpha of the input
-
df_dphi(r: <MagicMock id='140414711037064'>, phi: <MagicMock id='140414711053672'>)[source]¶ Gives the derivative of the nonlinearity with respect to the angle phi of the input
-
-
class
neuroptica.nonlinearities.AbsSquared(N)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityMaps z -> |z|^2, corresponding to power measurement by a photodetector.
-
__init__(N)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dphi(r: <MagicMock id='140414711116184'>, phi: <MagicMock id='140414711128696'>)[source]¶ Gives the derivative of the nonlinearity with respect to the angle phi of the input
-
-
class
neuroptica.nonlinearities.ComplexNonlinearity(N, holomorphic=False, mode='condensed')[source]¶ Bases:
neuroptica.nonlinearities.NonlinearityBase class for a complex-valued nonlinearity
-
__init__(N, holomorphic=False, mode='condensed')[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
backward_pass(gamma: <MagicMock id='140414711219424'>, Z: <MagicMock id='140414711244224'>) → <MagicMock id='140414711252640'>[source]¶ Backpropagate a signal through the layer :param gamma: backpropagated signal from the (l+1)th layer :param Z: output fields from the forward_pass() run :return: backpropagated fields delta_l
-
dIm_dIm(a: <MagicMock id='140414712026784'>, b: <MagicMock id='140414712039296'>) → <MagicMock id='140414712055904'>[source]¶ Gives the derivative of the imaginary part of the nonlienarity w.r.t. the imaginary part of the input
-
dIm_dRe(a: <MagicMock id='140414711985152'>, b: <MagicMock id='140414711997664'>) → <MagicMock id='140414712014272'>[source]¶ Gives the derivative of the imaginary part of the nonlienarity w.r.t. the real part of the input
-
dRe_dIm(a: <MagicMock id='140414711419232'>, b: <MagicMock id='140414711435840'>) → <MagicMock id='140414711972640'>[source]¶ Gives the derivative of the real part of the nonlienarity w.r.t. the imaginary part of the input
-
dRe_dRe(a: <MagicMock id='140414711377536'>, b: <MagicMock id='140414711385952'>) → <MagicMock id='140414711410816'>[source]¶ Gives the derivative of the real part of the nonlienarity w.r.t. the real part of the input
-
df_dIm(a: <MagicMock id='140414711335904'>, b: <MagicMock id='140414711348416'>) → <MagicMock id='140414711365024'>[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-
df_dRe(a: <MagicMock id='140414711294272'>, b: <MagicMock id='140414711302688'>) → <MagicMock id='140414711319296'>[source]¶ Gives the derivative of the nonlinearity with respect to the real part alpha of the input
-
df_dZ(Z: <MagicMock id='140414711265152'>) → <MagicMock id='140414711281760'>[source]¶ Gives the total complex derivative of the (holomorphic) nonlinearity with respect to the input
-
df_dphi(r: <MagicMock id='140414712105952'>, phi: <MagicMock id='140414712114368'>) → <MagicMock id='140414712139168'>[source]¶ Gives the derivative of the nonlinearity with respect to the angle phi of the input
-
-
class
neuroptica.nonlinearities.ElectroOpticActivation(N, alpha=0.1, responsivity=0.8, area=1.0, V_pi=10.0, V_bias=10.0, R=1000.0, impedance=<MagicMock name='mock.__rmul__()' id='140414711769016'>, g=None, phi_b=None)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityElectro-optic activation function with intensity modulation (remod).
This activation can be configured either in terms of its physical parameters, detailed below, or directly in terms of the feedforward phase gain, g and the biasing phase, phi_b.
If the electro-optic parameters below are specified g and phi_b are computed for the user.
alpha: Amount of power tapped off to PD [unitless] responsivity: PD responsivity [Watts/amp] area: Modal area [micron^2] V_pi: Modulator V_pi (voltage required for a pi phase shift) [Volts] V_bias: Modulator static bias [Volts] R: Transimpedance gain [Ohms] impedance: Characteristic impedance for computing optical power [Ohms]-
__init__(N, alpha=0.1, responsivity=0.8, area=1.0, V_pi=10.0, V_bias=10.0, R=1000.0, impedance=<MagicMock name='mock.__rmul__()' id='140414711769016'>, g=None, phi_b=None)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dIm(a: <MagicMock id='140414711827480'>, b: <MagicMock id='140414711844088'>) → <MagicMock id='140414711860696'>[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-
-
class
neuroptica.nonlinearities.LinearMask(N: int, mask=None)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityTechnically not a nonlinearity: apply a linear gain/loss to each element
-
__init__(N: int, mask=None)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
-
class
neuroptica.nonlinearities.SPMActivation(N, gain)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityLossless SPM activation function
phase_gain [ rad/(V^2/m^2) ] : The amount of phase shift per unit input “power”-
__init__(N, gain)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dIm(a: <MagicMock id='140414712214248'>, b: <MagicMock id='140414711706568'>) → <MagicMock id='140414711719080'>[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-
-
class
neuroptica.nonlinearities.Sigmoid(N)[source]¶ Bases:
neuroptica.nonlinearities.NonlinearitySigmoid activation; maps z -> 1 / (1 + np.exp(-z))
-
class
neuroptica.nonlinearities.SoftMax(N)[source]¶ Bases:
neuroptica.nonlinearities.NonlinearityApplies softmax to the inputs. Do not use in with categorical cross entropy, which implicitly includes this.
-
class
neuroptica.nonlinearities.bpReLU(N, cutoff=1, alpha=0)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityDiscontinuous (but holomorphic and backpropable) ReLU f(x_i) = alpha * x_i if |x_i| < cutoff f(x_i) = x_i if |x_i| >= cutoff
cutoff: value of input |x_i| above which to fully transmit, below which to attentuate alpha: attenuation factor f(x_i) = f-
__init__(N, cutoff=1, alpha=0)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
-
class
neuroptica.nonlinearities.cReLU(N)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityContintous, but non-holomorphic and non-simply backpropabable ReLU of the form f(z) = ReLU(Re{z}) + 1j * ReLU(Im{z}) see: https://arxiv.org/pdf/1705.09792.pdf
-
__init__(N)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dIm(a: <MagicMock id='140414710900384'>, b: <MagicMock id='140414710916992'>) → <MagicMock id='140414710409312'>[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-
-
class
neuroptica.nonlinearities.modReLU(N, cutoff=1)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityContintous, but non-holomorphic and non-simply backpropabable ReLU of the form f(z) = (|z| - cutoff) * z / |z| if |z| >= cutoff (else 0) see: https://arxiv.org/pdf/1705.09792.pdf (note, cutoff subtracted in this definition)
cutoff: value of input |x_i| above which to-
__init__(N, cutoff=1)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dphi(r: <MagicMock id='140414710825200'>, phi: <MagicMock id='140414710837712'>)[source]¶ Gives the derivative of the nonlinearity with respect to the angle phi of the input
-
-
class
neuroptica.nonlinearities.zReLU(N)[source]¶ Bases:
neuroptica.nonlinearities.ComplexNonlinearityContintous, but non-holomorphic and non-simply backpropabable ReLU of the form f(z) = z if Re{z} > 0 and Im{z} > 0, else 0 see: https://arxiv.org/pdf/1705.09792.pdf
-
__init__(N)[source]¶ Initialize the nonlinearity :param N: dimensionality of the nonlinear function :param holomorphic: whether the function is holomorphic :param mode: for nonholomorphic functions, can be “full”, “condensed”, or “polar”. Full requires that you specify 4 derivatives for d{Re,Im}/d{Re,Im}, condensed requires only df/d{Re,Im}, and polar takes Z=re^iphi
-
df_dIm(a: <MagicMock id='140414710476080'>, b: <MagicMock id='140414710488592'>) → <MagicMock id='140414710501104'>[source]¶ Gives the derivative of the nonlinearity with respect to the imaginary part beta of the input
-