MatrixMultOperator

public class MatrixMultOperator: ComputableOperator

matrix multiplication.

Transpose input tensors

Opertor’s attributes transposeA and transposeB indicating if tranposing input tensors before doing calculation. And if any or both of them are set to true, all caulcation and input/output validation will be doing after transposing.

Metal performance shader support

By default, operator tries to use MPSMatrixMultiplication in MetalPerformanceShaders. But on some devices which do not support MetalPerformanceShaders, we use self kernel.

Multiple input

This operator could takein multiple input. Currently, it support multiple input A and single input B. If inputTensors contains more than 2 elements, operator will view last element as input B and all previous element as input As.

Attributes

METAL_SUBMATRIX_SIZE
The submatrix size in Metal calcualtion. Should be the same as SUBMATRIX_SIZE in matrix_mult_op.metal
Declaration
Swift

public static let METAL_SUBMATRIX_SIZE = 4
Show on GitHub
operatorLabel
Operator label. Conforms to ComputableOperator
Declaration
Swift

public var operatorLabel: String
Show on GitHub
metalKernelFuncLabel
This operator does not operator on GPU. Conforms to ComputableOperator
Declaration
Swift

public var metalKernelFuncLabel:String = "MatrixMult"
Show on GitHub
computationDelegate
Conforms to ComputableOperator
Declaration
Swift

public var computationDelegate: OperatorCalculationDelegate?
Show on GitHub
inputTensors
Conforms to ComputableOperator
Declaration
Swift

public var inputTensors: [Tensor]?
Show on GitHub
outputTensors
Conforms to ComputableOperator
Declaration
Swift

public var outputTensors: [Tensor]?
Show on GitHub
transposeA
Whether transpose inputA before calculation
Declaration
Swift

public var transposeA: Bool
Show on GitHub
transposeB
Whether transpose inputB before calcualtion
Declaration
Swift

public var transposeB: Bool
Show on GitHub
disableInputOutputCheck
If true, operator will not call inputOutputTensorsCheck() before doing calculation. This is used inside framework to speed up in situation we know it will not be wrong.
Declaration
Swift

public var disableInputOutputCheck: Bool = false
Show on GitHub
trainable
Indicate if this operator would do paramter update.
Declaration
Swift

public var trainable: Bool = false
Show on GitHub
kernel
Kernel to choose
Declaration
Swift

public var kernel: MatrixMultKernel
Show on GitHub
mapType
The mapping type of this operator. Constant for this operator.
Declaration
Swift

public var mapType: OperatorMappingType
Show on GitHub
inPlaceble
Matrix multiplication cannot do in-place calculation
Declaration
Swift

public var inPlaceble: Bool = false
Show on GitHub
disabledMPS
If use MPS. Default is false
Declaration
Swift

internal var disabledMPS: Bool = false
Show on GitHub
matrixBeta
beta for mstrix calculation, if want to add into output tensor, set this to 1
Declaration
Swift

internal var matrixBeta: Float = 0.0
Show on GitHub

Initializer


                    
                    
                    init(operatorLabel:computationDelegate:transposeA:transposeB:inputTensors:outputTensors:kernel:disableInputOutputCheck:)

Undocumented

Declaration

Swift

public init(operatorLabel: String = "MatrixMultOperator",
            computationDelegate: OperatorCalculationDelegate? = nil,
            transposeA: Bool = false, transposeB: Bool = false,
            inputTensors: [Tensor]? = nil, outputTensors: [Tensor]? = nil,
            kernel: MatrixMultKernel = MatrixMultKernel.Single,
            disableInputOutputCheck: Bool = false)

Show on GitHub

Conforms to `ComputableOperator`


                    
                    
                    outputShape(shapeArray:)

Check the input shapes. Following same rule of matrix multiplication.

Note

This function will transpose shape first and then calcualte output shape.

Declaration

Swift

public func outputShape(shapeArray shapes: [TensorShape]) -> [TensorShape]?

Parameters


                                shapes

input shapes

Return Value

return shapes

Show on GitHub

inputOutputTensorsCheck()
Check if assigned inputTensors and outputTensors valid

Note

This function will first transposes input tensors’ shapes according to attributes transposeA and transposeB, and then validate the shapes.
Declaration
Swift

public func inputOutputTensorsCheck() -> (check: Bool, msg: String)
Return Value

the result and erro message if possible

Show on GitHub


                    
                    
                    compute(_:)

Compute sync

Declaration

Swift

public func compute(_ computationMode: OperatorComputationMode = SerranoEngine.configuredEngine.defaultComputationMode)

Parameters


                                computationMode

mode

Show on GitHub


                    
                    
                    computeAsync(_:)

Compute async

Declaration

Swift

public func computeAsync(_ computationMode: OperatorComputationMode = SerranoEngine.configuredEngine.defaultComputationMode)

Parameters


                                computationMode

mode

Show on GitHub


                    
                    
                    gradCompute(_:)

Calulate grads sync. All unary operator return grads tensor with same number and shape as attribute inputTensors.

Declaration

Swift

public func gradCompute(_ computationMode: OperatorComputationMode) -> [String: DataSymbolSupportedDataType]

Parameters

`computationMode`	computationMode
`upGrds`	upGrds

Return Value

return grads tensor

Show on GitHub


                    
                    
                    gradComputAsync(_:)

Cal grads async

Declaration

Swift

public func gradComputAsync(_ computationMode: OperatorComputationMode)

Parameters

`computationMode`	computationMode
`upGrds`	upGrds

Show on GitHub


                    
                    
                    updateParams(grads:LR:)

Update params if possible. No update parameters for binary operators.

Declaration

Swift

public func updateParams(grads: [Tensor], LR: Float)

Parameters

`grads`	grads tensor list
`LR`	learning rate

Show on GitHub

bindParamSymbols(_:)
Do nothing
Declaration
Swift

public func bindParamSymbols(_ symbols: [GraphSymbol])
Show on GitHub
paramSymbols()
This operator has no parameters.
Declaration
Swift

public func paramSymbols() -> [GraphSymbol]
Return Value

An empty array

Show on GitHub
MNKFetch(tensorA:tensorB:)
Get M, N, K attributes
Declaration
Swift

internal func MNKFetch(tensorA: Tensor, tensorB: Tensor) -> (M: Int, N: Int, K: Int)
Return Value

M, N, K

Show on GitHub
cpu()
Use BLAS cblas_sgemm to do calculation
Declaration
Swift

internal func cpu()
Show on GitHub
gpu()
This method choose proper kernel or MPS to do calculation
Declaration
Swift

internal func gpu()
Show on GitHub


                    
                    
                    gpu_single(inputABuffer:inputBBufferTransposed:outputCBuffer:dimInfo:kernel:)

Do calculation of inputA and transpoedB. transposedInputB is supposed already transposed

Declaration

Swift

internal func gpu_single(inputABuffer: MTLBufferResource, inputBBufferTransposed: MTLBufferResource, outputCBuffer: MTLBufferResource,
                         dimInfo: inout MatrixDimInfo, kernel: MTLComputePipelineState)

Parameters

`inputABuffer`	inputA
`inputBBufferTransposed`	transposedInputB
`outputCBuffer`	outputCBuffer
`dimInfo`	dimInfo
`kernel`	kernel

Show on GitHub

gpu_kernel_single()
Undocumented
Declaration
Swift

internal func gpu_kernel_single()
Show on GitHub


                    
                    
                    gpu_submatrix(inputATransposeBuffer:inputBBuffer:outputCBuffer:dimInfo:kernel:)

Do matrix multiplication with submatrix kernel.

Declaration

Swift

internal func gpu_submatrix(inputATransposeBuffer: MTLBufferResource, inputBBuffer: MTLBufferResource,
                            outputCBuffer: MTLBufferResource,
                            dimInfo: inout MatrixDimInfo, kernel: MTLComputePipelineState)

Parameters

`inputATransposeBuffer`	transposed A buffer
`inputBBuffer`	inputBBuffer
`outputCBuffer`	outputCBuffer
`dimInfo`	dimInfo
`kernel`	kernel

Show on GitHub

gpu_kernel_submatrix()
Note
There’s no any transposing processing in this function cause in function gpu() it only dispatches suitable inputs to this function.
Declaration
Swift

internal func gpu_kernel_submatrix()
Show on GitHub
gpu_kernel_MPS()
Undocumented
Declaration
Swift

internal func gpu_kernel_MPS()
Show on GitHub