View on GitHub
Install in Dash
FullyconnectedOperator Class Reference
FullyconnectedOperator
public class FullyconnectedOperator: ComputableOperatorThe regular fully connected operator.
Operator do the 1D_dot(inputTensor.flatten, weights) + bias calculation on all input tensors.
Weight tensor layout
The weight is a 2D tensor with shape: [n, m] where :
n: the flattened dimension ofinputTensors, i.e. same value ascountof a input tensor.m: the number of hidden nodes, same value asnumUnits;
Thus, each column stores the weights of corresponding hidden unit.
Bias tensor layout
The bias is a 1D tensor with shape [m] where:
m: the number of hidden nodes, same value asnumUnits;
Thus, each value in the tensor is the bias value of corresponding hidden unit.
Input tensors auto flatten
For input tensor with rank >=2, the operator will automatically flatten the tensor and then do calcualtion.
Multiple input tensors
If inputTensors has more than 1 tensor object,
the operator applies calculation on each input tensor independently
and stores the results in corresponding tensor of outputTensors.
Note
All input tensor should have samecount.
Bias enable choice
Bias could be disabled by setting the biasEnabled to false.
Batch calculation
This operator itself does not explicitly support batch calculation. But user can use slice tensor to do the same thing. Details can be found in Slice tensor and Batch calculation with operators
-
Operator label. Conforms to
ComputableOperatorDeclaration
Swift
public var operatorLabel: String -
This operator does not operator on GPU. Conforms to
ComputableOperatorDeclaration
Swift
public var metalKernelFuncLabel:String = "Fullyconnected" -
Conforms to
ComputableOperatorDeclaration
Swift
public var computationDelegate: OperatorCalculationDelegate? -
Conforms to
ComputableOperatorDeclaration
Swift
public var inputTensors: [Tensor]? -
Conforms to
ComputableOperatorDeclaration
Swift
public var outputTensors: [Tensor]? -
If use
bias. Default istrue.Declaration
Swift
public var biasEnabled: Bool = true -
Weight tensor.
Shape specific
The tensor should be with shape
[inputDim, numUnits].Note
Ifweightisnilwhen calculation,fataErrorwill be raised.Declaration
Swift
public var weight: Tensor? -
Bias tensor
Shape specific
The tensor should be with shape
[numUnits].Note
Ifbiasisnilwhen calculation,fataErrorwill be raised.Declaration
Swift
public var bias: Tensor? -
Number of input units. Must be a positive integer.
Declaration
Swift
public var inputDim: Int = 1 -
Number of hidden units. Must be a positive integer.
Declaration
Swift
public var numUnits: Int = 1 -
If
true, operator will not check theupGrads‘s shape. This is used inside framework to speed up in situation we know it will not be wrong. Cases like auto generated differentiation graph.Declaration
Swift
public var disableUpGradShapeCheck: Bool = false -
If
true, operator will not callinputOutputTensorsCheck()before doing calculation. This is used inside framework to speed up in situation we know it will not be wrong.Declaration
Swift
public var disableInputOutputCheck: Bool = false -
Indicate if this operator would do paramter update.
Declaration
Swift
public var trainable: Bool = true -
The mapping type of this operator.
OneToOnefor this operator.Declaration
Swift
public var mapType: OperatorMappingType -
fully connected operator cannot do in-place calculation
Declaration
Swift
public var inPlaceble: Bool = false -
If disable using MPS
Declaration
Swift
public var disabledMPS: Bool = false
-
Designated init
Declaration
Swift
public init(inputDim: Int, numUnits: Int, operatorLabel: String = "FullyconnectedOperator", inputTensors: [Tensor]? = nil, outputTensors: [Tensor]? = nil, computationDelegate: OperatorCalculationDelegate? = nil, weight: Tensor? = nil, bias: Tensor? = nil)Parameters
inputDiminputDim
numUnitsnumUnits
operatorLabeloperatorLabel
inputTensorsinputTensors
outputTensorsoutputTensors
computationDelegatecomputationDelegate
weightweight
biasbias
-
Declaration
Swift
public func outputShape(shapeArray shapes: [TensorShape]) -> [TensorShape]?Parameters
shapesinput shapes
Return Value
result shapes
-
Check validation of
inputTensors,outputTensors,weightandbias.Declaration
Swift
public func inputOutputTensorsCheck() -> (check: Bool, msg: String)Return Value
check, if pass. msg, error message.
-
Compute synclly.
Declaration
Swift
public func compute(_ computationMode: OperatorComputationMode = SerranoEngine.configuredEngine.defaultComputationMode)Parameters
tensorsinput tensors
computationModecmputation mode. If choose
GPUbut haven’t configued a GPU SerranoEngine, operator will useCPUto compute.Return Value
result tensors
-
Compute asynclly
Declaration
Swift
public func computeAsync(_ computationMode: OperatorComputationMode = SerranoEngine.configuredEngine.defaultComputationMode)Parameters
tensorsinput tensors
computationModecomputation mode
-
Calulate grads sync. All unary operator return grads tensor with same number and shape as attribute
inputTensors.Declaration
Swift
public func gradCompute(_ computationMode: OperatorComputationMode) -> [String: DataSymbolSupportedDataType]Parameters
computationModecomputationMode
Return Value
return grads tensor
-
Cal grads async
Declaration
Swift
public func gradComputAsync(_ computationMode: OperatorComputationMode)Parameters
computationModecomputationMode
-
Update params if possible. No update parameters for binary operators.
Declaration
Swift
public func updateParams(grads: [Tensor], LR: Float)Parameters
gradsgrads tensor list
LRlearning rate
-
Bind according to labels.
-Note: if cannot bind all needed parameters.
fatalErrorwill be raised.Declaration
Swift
public func bindParamSymbols(_ symbols: [GraphSymbol]) -
Attribute
weightas aTensorSymbol. Attributebiasas aTensorSymbol.Declaration
Swift
public func paramSymbols() -> [GraphSymbol]Return Value
Array of GraphSymbol