o hm@svdZddlZddlZddlmZddlmZmZddlm Z m Z m Z m Z ddl Z ddlmZddlmZddlmZe eeefeeeeeffe fZd e_e Ze e ee e fee ee e ffe fZd e_Gd d d Ze jje jje jje jje jje jj e jj!e jj"e jj#e jj$e jj%e jj&e jj'e jj(e jj)hZ*e jj+j,e jj+j-e jj+j.e jj+j/e jj+j0e jj+j1e jj+j2e jj+j3e jj+j4e jj+j5e jj+j6e jj+j7e jj+j8e jj+j9e jj+j:e jj+j;e jj+je j?e j@e j>e jAe jBe jCe jDe jEe jFhZGe jHd d ddddddddddddddddddd d!d"d#hZId$d%ZJd&d'ZKd(d)ZLd*d+ZMd,e d-eNd.e fd/d0ZOd1d2ZPd3d4ZQd5d6ZRd7d8ZSd9d:ZTd;d<ZUd=d>ZVd?d@ZWdAdBZXdCdDZYdEdFZZd.e[fdGdHZ\dIdJZ]dKdLZ^dMe j_dNe j_d.e[fdOdPZ`dQeadReadSe[dTe jbdUe[d.eeaeaff dVdWZcdXdYZddZd[Zed\e jjfd]e jjfd.e eNfd^d_Zgd`e daeheNe fd.eheNe ffdbdcZid`e d.ddfdedfZjdge daeheNe fd.ddfdhdiZkdQeadRead.dfdjdkZle jmfdMe j_dNe j_dQeadReadTe jbdle j_dSe[dme jnd.ee j_e j_ffdndoZod`e d.eafdpdqZpdre jjfdsee dtfd.eheNee dtfffdudvZqdwe jjfd.e fdxdyZrgdzZsdS){z? Utils shared by different modes of quantization (eager/graph) N) OrderedDict)getfullargspec signature)AnyCallableOptionalUnion) QuantType)Node)is_parametrizedztorch.ao.quantization.utilsc@seZdZdZdS) MatchAllNodeznA node pattern that matches all nodes, used in defining fusion patterns in FX Graph Mode Quantization N)__name__ __module__ __qualname____doc__rro/var/www/html/construction_image-detection-poc/venv/lib/python3.10/site-packages/torch/ao/quantization/utils.pyr %sr relurelu_ contiguousdetachdetach_ hardsigmoid hardsigmoid_permuterepeatrepeat_interleavereshaperesize_shapesigmoidsigmoid_sizesqueezesqueeze_tanhtanh_ transpose unsqueeze unsqueeze_viewcCsR|jdko |jtv}|jdko|jtv}|jdko#t|t|jtv}|||fS)N call_function call_method call_module)optarget func_list method_listtypestrmodule_type_list)nodemodulesis_call_functionis_call_methodis_call_modulerrr check_nodexs  r:cCs|}|||S)a Combines two dictionaries. This function takes two dictionaries as input and returns a new dictionary that contains all the key-value pairs from both input dictionaries. If there are any duplicate keys in the `additional_dict`, the values from the `additional_dict` will overwrite those in the `default_dict`. Args: default_dict (dict): The main dictionary that will be used as the base additional_dict (dict): The dictionary used to update `default_dict` Returns: dict: The resulting dictionary Example: >>> x = dict(a=1, b=1) >>> y = dict(b=2, c=3) >>> get_combined_dict(x, y) {'a': 1, 'b': 2, 'c': 3} )copyupdate) default_dictadditional_dictdrrrget_combined_dicts r@cCs|tjkp |tjkSN)torchper_tensor_affineper_tensor_symmetricqschemerrr is_per_tensorsrGcCs|tjtjtjfvSrA)rBper_channel_affine per_channel_affine_float_qparamsper_channel_symmetricrErrris_per_channels rKobjfqnreturncCstt|d|S)zO Given an obj and a fqn such as "foo.bar.baz", returns gm.foo.bar.baz. .) functoolsreducegetattrsplit)rLrMrrrgetattr_from_fqnrTcCstjtjtjtjtjtjtjtjtjtjtjtjtjtjtj tj tj tj tjtjtj tj tj tj i }||vs>Jdt |||S)NzUnsupported dtype: )rBquint8uint8qint8int8qint32int32quint4x2quint2x4uint16int16 float8_e5m2 float8_e4m3fnr3)qdtype DTYPE_MAPPINGrrrto_underlying_dtypesrdcCsddlm}t|dd}|j}||d}|rt||r d|dSt|r(tj}nt|r:|tj kr4tj }|j |d<nt d|||d<| \}}||d<||d<t|d r]|j|d <t|d rg|j|d <|S) Nr)PlaceholderObserverrF)rFdtypeaxiszUnrecognized qscheme: scale zero_point quant_min quant_max)torch.ao.quantization.observerrerRrf isinstancerGrBrCrKrJrHch_axis RuntimeErrorcalculate_qparamshasattrrjrk)observer_or_fake_quantrerFrfqparamsrhrirrrget_qparam_dicts,           rtcCsDt|}||i}t||vsJdt|d||t|S)aGet the observed/quantized custom module class that we need to swap `custom_module` to Input: custom_module: input, can be an instance of either a float or observed custom module custom_module_class_mapping: the float to observed or observed to quantized custom module class mapping qconfig: qconfig configured for the custom module Output: corresponding observed/quantized custom module class for input custom module instance z5did not find corresponding observed module class for z in mapping: )get_quant_typegetr2) custom_modulecustom_module_class_mappingqconfig quant_type class_mappingrrrget_swapped_custom_module_classs  r|cC|dusJ|}|jSrA) activationrf)ryr~rrractivation_dtype rcCr}rA)weightrf)ryrrrr weight_dtyperrc Cs>t|tjtjtjtjtjtjtjtj tj tj f vot | S)zGiven a qconfig, decide if the activation needs to be quantized or not, this includes quantizing to quint8, qint8 and qint32 and float16 ) rrBrVrXrZfloat16rWrYr_r[r`ra#activation_is_dynamically_quantizedryrrr"activation_is_statically_quantizeds rcCst|\}}}|S)zGiven a qconfig, decide if the activation needs to be dynamically quantized or not, this includes dynamically quantizing to quint8, qint8 and float16 )get_qconfig_dtypes)ry_activation_dtype_activation_is_dynamicrrrrsrcCt|tjtjtjtjfvS)zGiven a qconfig, decide if the activation needs to be quantized to int8 or not, this includes quantizing to quint8, qint8 )rrBrVrXrWrYrrrractivation_is_int8_quantized"s rcCst|tjtjfvS)zXGiven a qconfig, decide if the activation needs to be quantized to int32 or not )rrBrZr[rrrractivation_is_int32_quantized.rUrc Cs4t|tjtjtjtjtjtjtjtj tj tj f vS)zKGiven a qconfig, decide if the weight needs to be quantized or not ) rrBrVrXrr\rWrYr_r[r`rarrrrweight_is_quantized5srcCr)zVGiven a qconfig, decide if the weight needs to be statically quantized or not )rrBrVrXrWrYrrrrweight_is_statically_quantizedGsrcCs2t|\}}}|tjtjfvo|tjtjfvo|S)zTGiven a qconfig, returns True if this op is using int8 dynamic quantization )rrBrVrWrXrY)ryrrrrrr op_is_int8_dynamically_quantizedNs rcCs6|dusJ|}|}t|dd}|j|j|fS)zgreturns the qconfig tuple for qconfig: (activation_dtype, weight_dtype, activation_is_dynamic) N is_dynamicF)r~rrRrf)ryr~ract_is_dynamicrrrr\s  rc Cs|dusJ|}|}tjtjtjtjtjtjtj tj tj tj g }|j |vr?t|dr4|jr4tjS|j |vr> class M(torch.nn.Module): def __init__(self) -> None: self.linear = torch.nn.Linear(5, 5) def forward(self, x): return self.linear(x) >> m = M() >> l = m.linear >> _get_path_of_module(m, l) "linear" N) named_modules)rrnprrr_get_path_of_modules rfloccsfdd|DS)zGet local keyword arguments Example:: >> def f(self, a, b=9): pass >> loc = {"a": 6, "c": 7} >> _get_signature_locals(f, loc) {"a": 6} cs$i|]\}}|tjvr||qSr)r parameters).0kvrrr "s$z)_get_signature_locals..)items)rrrrr_get_signature_localss rzOrderedDict[str, Any]cCsfi}t|jD]%\}}|j|jur|j||<q |j|jur$d||<q |j|jur.i||<q t|S)zGet all default keyword arguments from function signature Example:: >> def f(self, a, b=9): pass >> _get_default_kwargs(f) {"b": 9} r) rrrdefaultemptykindVAR_POSITIONAL VAR_KEYWORDr)rkwargsnameparamrrr_get_default_kwargs%s      rfunccCs@t|}t||}|}|D] \}}||vr|||<q|S)a(Given a function and local function arguments, normalize the keyword arguments by filling in default arguments from function signature Example:: >> def f(self, key1=3, key2=3): pass >> loc = {"key2": 6} >> _normalize_kwargs(f, loc) {"key1": 3, "key2": 6} )rrr;r)rrdefault_kwargs local_kwargsnormalized_kwargsattrvalrrr_normalize_kwargs:s rcCs8|dkr |ksJdJd||ksJddS)aeValidates that the user-specified quantization range is properly initialized and within the given bound supported by the observer dtype. To accommodate lower-bit quantization with respect to the existing torch.qint8 and torch.quint8 datatypes, the user can choose to use dynamic quantization range by passing in a tuple of initial qmin and qmax values. One use case is these customized qmin and qmax values are used to calculate static estimates of the scale and zero point for aggressive lower-bit fake quantization. These estimates are compared against parameters learned through backpropagation. The related literatures for scale and zero point via backpropagation are as follows: Learned Step Size Quantization: https://openreview.net/pdf?id=rkgO66VKDS Trained Quantization Thresholds: https://arxiv.org/pdf/1903.08066.pdf rz1Used-specified quantization range must include 0.zKqmin must be strictly less than qmax for user-specified quantization range.Nr)rjrkrrrvalidate_qmin_qmaxPs repsrFc Cs*t||stjdg|jjdtjdg|jjdfSt|t|}t|t|} |j} tj| tj | d} tj | tj | d} | | }|tjksS|tjkrt| | } | t||d} t| |} |tjtjfvr|r| | ||d} nO| | d} nF|tjkr||t||} t| |k| t| } d|| } n$| |t||} t| |} |t||  tj} t| ||} t| jdkrtjt| g| j| d} t| jdkr tjt| g| j| d} |tjkr tjt| g| j| d} | tj | tj fS)adCalculates the quantization parameters, given min and max value tensors. Works for both per tensor and per channel cases Args: min_val: Minimum values per channel max_val: Maximum values per channel Returns: scales: Scales tensor of shape (#channels,) zero_points: Zero points tensor of shape (#channels,) g?devicer)rfrr)rrBtensorrr2min zeros_likemaxonesr"doublezerosint64torDrJrrWrVnew_fullrIwhere ones_likeroundintclamprrrf) rrrjrkrfrrrF min_val_neg max_val_posrrhrirrrdetermine_qparamslsP       rcCstt|jS)zGet number of positional args for a function Example:: >> def f(self, key1=3, key2=3): pass >> _get_num_pos_args(f) 3 )rrargsrrrr_get_num_pos_argss rmodelexample_inputs.csR|ifdd}tjjj|tjj_z ||Wtjj_Stjj_w)aGiven a model and its example inputs, return a dictionary from fully qualified name of submodules to example_inputs for that submodule, e.g. {"linear1": (tensor1,), "linear2": (tensor2,), "sub": (tensor3,), "sub.linear1": (tensor4,), ...} Used to make quantizing submodules easier now that FX Graph Mode Quantization requires example inputs. Also works for keyword arguments with default values, we would flatten keyword arguments as positional arguments and fill in the missing keyword args with default values, e.g. if we have a forward function: def forward(self, x, key1=3, key2=3): ... and we call it with self.submodule(x, key2=6) we'll get example_inputs: (x, 3, 6) user can also override `key1` with positional arguments as well: for self.submodule(x, 5, key2=6) we'll get: (x, 5, 6) variable positional arguments and variable positional keyword arguments in forward function are not supported currently, so please make sure no submodules is using them. c st|}t|j|}t|jd}|t|}|r+|r+|jdd|d8}|r+|s||t |}t |}|durC||<|g|Ri|S)NrF)last) listr;rforwardrrpopitemextendvaluestupler) selfrrsubmodule_example_inputsrnum_args num_to_popsubmodule_example_inputs_tuplerMfqn_to_example_inputsorig_module_callrrr_patched_module_calls     z7get_fqn_to_example_inputs.._patched_module_call)rBnnModule__call__)rrrrrrget_fqn_to_example_inputss     r rcCsdd|Ddd|DB} tdtdh|kr*tdtdh} t|dks8Jd|t|dkrFtt|}|Sd }|S) z Returns the unique device for a module, or None if no device is found. Throws an error if multiple devices are detected. cSsh|]}|jqSrr)rrrrr sz0_assert_and_get_unique_device..cpumetazfBoth 'meta' and 'cpu' are present in the list of devices. Module can have one device. We Select 'cpu'.rzKprepare only works with cpu or single-device CUDA modules, but got devices rN) rbuffersrBrrrrnextiter)rdevicesrrrr_assert_and_get_unique_devices& r) NodePatternPatternr r:r@rGrKrTrtr|rrrrrrrrrrrurrrr rdrr)trrPr collectionsrinspectrrtypingrrrrrB torch.ao.quantization.quant_typer torch.fxr torch.nn.utils.parametrizer rrr QuantizerClsrr rReLUReLU6AdaptiveAvgPool1dAdaptiveAvgPool2dAdaptiveAvgPool3d AvgPool1d AvgPool2d AvgPool3d MaxPool1d MaxPool2d MaxPool3dIdentity HardsigmoidSigmoidTanhr4 functionaladaptive_avg_pool1dadaptive_avg_pool2dadaptive_avg_pool3delu hardswish instance_norm layer_norm leaky_relusilumishdropout max_pool1d max_pool2d max_pool3drhardtanh hardtanh_rr r'rr#stacksumr%r(catr0meanr1r:r@rGrKr3rTrdrtr|rrrrrrrrboolrrruTensorrrrfrrrrrdictrrrrrCrFrrr r__all__rrrrsJ    &" $   $  >  &$ N   9