gras 1.1.2

A Genetic Programming framework for Neural Architecture Search in Rust
Documentation
�

�ݲj~2����dZddlZddlZddlmZmZddlmZddlZddl	m
Z
eGd�d��ZeGd�d	��Zd
e
ezde
deeeffd
�Zdd�Zedk(re�yy)u�Load a gras-evolved net into pure PyTorch — no gras, no Rust needed.

gras exports every trained net as two artifacts in the run dir:

  nets/<full-hash>.json          — the topology (the "how to build it")
  elite-<short-hash>.safetensors — the trained weights (or nets/<hash>.json
                                   snapshots for non-elites, same format)

The safetensors naming convention has **two families**, and BOTH must be
loaded (see the note at the bottom for why):

  node<N>.weight / node<N>.bias              — one Linear per graph node
  proj<N>_<port>_<src>.weight / .bias        — one Linear per cross-dim bridge
                                               (a wire whose source and target
                                               node dims differ)

Requirements: `pip install safetensors torch` (numpy optional). Test:

  python examples/python/load_gras_net.py results/<run_dir> <hash-prefix>
�N)�	dataclass�field)�Path)�	load_filec��eZdZUeed<eed<dZeed<dZeed<dZeed<dZ	eed<d	Z
ed	zed
<dZeed<d	Z
ed	zed
<y	)�Node�id�kindr�
num_inputs�num_outputs�
hidden_dim�Identity�
activationN�port_activations�Add�
combine_op�standardize)�__name__�
__module__�__qualname__�int�__annotations__�strrrr
rr�listrr���1/home/maulb/gras/examples/python/load_gras_net.pyrr"s[���G�

�I��J����K����J��� �J�� �$(��d�T�k�(��J���"�K��t��"rrc��eZdZUeed<eed<eeed<eeed<ee��Z	eed<ee��Z
eed<d
d
�Zdejdejfd�Zy	)�GrasNet�	input_dim�
output_dim�nodes�connections)�default_factory�layers�	wire_proj�returnNc����|jD�cic]}|j|j��c}�t�j	��}|jD�cic]}|jg��}}|j
D]"}||ddj
|dd��$i}|jD]o}|jdk(r
|j}n3||js|}n!t�fd�||jD��}|�|jf||j<�q||_	tjj�|_
|jD]N}||j\}}tjj||�|jd|j��<�Pi|_i}t!|j
�D]�\}	}|dd|dd}}
|dd}|j#|
|fd�}
|
dz||
|f<||d||
dk7s�Nd	|
�d
|�d
|
��}||j|	<tjj||d||
d�|j|<��ycc}wcc}w)zyOne torch.nn.Linear per node + one per cross-dim bridge, named
        EXACTLY as gras's safetensors exporter names them.�to�node�from�Inputc3�(�K�|]	}�|���y�w�Nr)�.0�s�
out_dim_ofs  �r�	<genexpr>z(GrasNet.build_modules.<locals>.<genexpr>Qs�����C�1B�A�*�Q�-�1B�s��indexr��proj�_N)r"r	r
�max�valuesr#�appendr
r �dims�torch�nn�
ModuleDictr%�Linearr&�	enumerate�get)�self�n�eff�all_sources�cr:�ind�outd�arrivals�ci�t�tpr0�src_idx�namer1s               @r�
build_moduleszGrasNet.build_modules;sO���37�*�*�=�*�Q�a�d�d�A�L�L�(�*�=�
��*�#�#�%�&��?C�z�z�,J�z�!�Q�T�T�2�X�z��,J��!�!�A���$����(�/�/��&�	�&�0A�B�"������A��v�v�� ��n�n�� ����&����C��Q�T�T�1B�C�C���z�!�$�$�/�0�D����J����	��h�h�)�)�+������A��Q�T�T�
�I�C��).������d�)C�D�K�K�$�q�t�t�f�
�&�����/1���t�/�/�0�E�B���d�G�F�O�Q�t�W�W�%5�r�A��&�	�&�!�A��l�l�A�r�7�A�.�G� '�!��H�a��W���A�w�q�z�T�!�W�Q�Z�'��a�S��"��Q�w�i�0��%)����r�"�$)�H�H�O�O���G�A�J��Q���
�%����D�!�1��A>��,Js�I(�I-�xc���idtjj��dtjj��dtjj	��dtjj��dtjj
��dtjj��dtjj��dtjj��d	tjjd
��dtjj��dtjj	d
���dtjj��dtjj��dtjj��dtj�dtj ��dt"dtj$dtj$f�fd�}dt"dzdtj$dtj$fd�}i}i}t'|j(�D]=\}}|dd|ddf}|j+|d�}	|	dz||<|d|d|	f||<�?i}
|j,D�]�}|j.dk(rR|j0d|j2��|�}t5|j6xsd�D]}
||
|j2|
f<��et'|j(�D��cgc]$\}}|dd|j2k(r
||d df��&}}}g}|D]j\}}|j(|}|
|d d|d df}|j8j+|�}|�|j0||�}|j;|��l|r�|j<xsd!}|d!k(r|d}|ddD]}||z}�	n�|d"k(r(tj>|d�#�jAd�#�}n�|d$vr_|d}|ddD]Q}|d%k(r||znD|d&k(r||z
n:|d'k(r||zn0|d(k(rtjB||�ntjD||�}�Sn>tGd)|����tjHd|jJ|j2d�}|j0d|j2��|�}||jL|�}t5|j6xsd�D]Y}
|jNr'|
tQ|jN�kr|jN|
n|jR}|||�|
|j2|
f<�[���tUd*�|j,D��}|
|j2dfScc}}w)+u�x: [batch, input_dim] → [batch, output_dim].

        Faithful mirror of Network::forward:
          per node (id order): gather wires flat in (port, source) order →
          combine (Add/Mean/Multiply/Subtract/Divide/Max/Min over the FLAT
          sequence) → node Linear → standardize → per-port activation fan-out.
        Input nodes' ports stay RAW (no activation — validation rule).
        r�ReLU�GeLU�SiLU�SELU�Tanh�Sigmoid�Mish�	LeakyReLUg{�G�z�?�ELU�GeluTanh�tanh)�approximate�Softplus�	HardSwish�HardSigmoid�Sin�CosrMrJr'c����|dk(rtj|d��S|dk(rtj|d��S�j|�}|�t	d|�d���||�S)N�Softmax�������dim�
LogSoftmaxzunknown activation u — extend act_map)r;�softmax�log_softmaxr@�
ValueError)rMrJ�f�act_maps   �r�	apply_actz"GrasNet.forward.<locals>.apply_act�si����y� ��}�}�Q�B�/�/��|�#��(�(���3�3����D�!�A��y� �#6�t�h�>Q�!R�S�S��Q�4�KrNc�4�|dvr|S|dk(s|dk(r;tjjj||jdfddd�S|dk(r;|tj
|j
d�jdd�	�dz�zStd
|����)N)Nr�	LayerNorm�InstanceNormrdg�h㈵��>�RmsNorm�T)�keepdimzunknown standardize op )	r;r<�
functional�
layer_norm�shape�rsqrt�pow�meanrj)rMrJs  rrz$GrasNet.forward.<locals>.standardize�s����)�)����{�"�d�n�&<��x�x�*�*�5�5�������~�t�T�4����y� ��5�;�;�q�u�u�Q�x�}�}�R��}�'F��'M�N�N�N��6�t�h�?�@�@rr)r*r3rr4r,r+r�Meanre)�Multiply�Subtract�Divide�Max�Minr{r|r}r~zunknown combine op c3�@K�|]}|jdk(s�|���y�w)�OutputN)r
)r/rBs  rr2z"GrasNet.forward.<locals>.<genexpr>�s����D�:�a����8�1C��:�s��)+r;r<rrQ�GELUrSrTrUrVrWrXrYr]�	Hardswish�Hardsigmoid�sin�cosr�Tensorr?r#r@r"r
r%r	�rangerr&r9r�stackry�maximum�minimumrj�zerosr:rr�lenr�next)rArOrmr�wire_atrHrIrE�key�pos�port_outputsrB�base�p�wires�tensors�_s�sigrM�op�zrJ�a�out_noderls                        @r�forwardzGrasNet.forwardrsK���
�����)�)�+�
��E�H�H�M�M�O�
�
�E�H�H�M�M�O�
�
�E�H�H�M�M�O�	
�

�E�H�H�M�M�O�
�
�E�H�H�M�M�O�

�
�u�x�x�'�'�)�
�
�E�H�H�M�M�O�
�
����+�+�D�1�
�
�5�8�8�<�<�>�
�
����
�
�&�
�9�
�
����)�)�+�
�
����+�+�-�
�
�5�8�8�/�/�1�
�
�5�9�9�
� 
�5�9�9�!
��&	�C�	�E�L�L�	�U�\�\�	�	A�c�D�j�	A�U�\�\�	A�e�l�l�	A� 46��/1���t�/�/�0�E�B���T�7�6�?�A�d�G�G�$4�5�C��,�,�s�A�&�C��!�G�H�S�M��q�6�3�q�6�3�/�G�B�K�	1�=?�����A��v�v�� �1�t�{�{�T�!�$�$��=�1�!�4���q�}�}�1��2�A�.2�L�!�$�$���+�3��
'�t�'7�'7�8�,�8�E�B���T�7�6�?�a�d�d�*��Q�v�Y�v�&�'�8�
�,�
�G����B��$�$�R�(��"�A�f�I�f�$5�q��y��7I�#J�K���~�~�)�)�"�-���#�+�$�+�+�d�+�C�0�C����s�#�
 ���\�\�*�U����;���
�A�$�Q�R�[����E��)��6�\����G��3�8�8�Q�8�?�A��K�K���
�A�$�Q�R�[��%'�:�%5�A��E�*,�
�*:��Q��*,��.��Q��8:�e�����q�!�!4�!&���q�!�!4��)�%�':�2�&�%A�B�B��K�K��4�9�9�Q�T�T�?�1�#5�6��-�4�;�;��a�d�d�V�}�-�a�0�D��q�}�}�d�3�D��1�=�=�-�A�.���)�)�a�#�a�6H�6H�2I�.I��&�&�q�)�����
+4�A�t�*<��a�d�d�A�Y�'�
/�a�p�D�4�:�:�D�D���X�[�[�!�,�-�-��a,s�)V	�r'N)rrrrrrr�dictrr%r&rNr;r�r�rrrrr/sb���N��O���:���d�����.�F�D�.��D�1�I�t�1�3�ny.����y.�%�,�,�y.rr�run_dir�hash_prefixr'c��t|�}t|jd|�d��d�}|�td|�d|����t	j
|j
��}t	j
|d�}|d}t|d|d	|d
D���cgc]/}tdi|j�D��cic]\}}||��
c}}����1c}}}|d��}	|	j�|d
dd}
d}dD] }||�|
�d�z}
|
j�s�|
}n|�td|
�d|
�d|�d���tt|��}|	jj�}|D�cgc]	}||vs�|��}}|rt!dt#|��d|dd�d���|	jj%|D�cic]}|||��
c}d��|	|fScc}}wcc}}}wcc}wcc}w)u�Rebuild the net from nets/<hash>.json and fill its trained weights
    from elite-<short>.safetensors. Raises if any parameter is missing —
    a partial load is a different network, never accept one silently.znets/z*.jsonNzno nets/z
*.json in �topology�optionsr r!r"r#)r r!r"r#�hash�)zelite-zworst-z.safetensorsz	no elite-z.safetensors or worst-z.safetensors in u� — is this net an exported elite/worst? (non-elite nets exist only as topology snapshots in nets/, their weights are not saved)zpartial safetensors: z parameter(s) absent (e.g. �u&) — refusing a silently crippled netT)�strictr)rr��glob�FileNotFoundError�json�loads�	read_textrr�itemsrN�existsrrr%�
state_dictrjr��load_state_dict)r�r��
state_path�state�topo�optsrB�k�v�net�short�weights_path�prefixr�r��sd�missings                 r�
load_gras_netr��s#���7�m�G��g�l�l�U�;�-�v�#>�?��F�J����(�;�-�z�'�� K�L�L��J�J�z�+�+�-�.�E��:�:�e�J�'�(�D��	�?�D�
��{�#���%�=A�'�]�K�]��t�3����	�2�	���1�q�!�t�	�2�3�]�K���'�	�C������&�M�"�1��E��L�&�������|�4�4���8�8�:��L��	'�
�����w�4�U�G�;K�G�9�UK�
L�
�	
�
��L�)�*�G�	���	�	�	 �B��1�"�Q��� 0�q�"�G�1���#�C��L�>�2��R�a�[�M�!G�
I�
�	
��J�J���r�:�r�!��7�1�:�
�r�:�4��H���:���=3��K��.2�� ;s*�G�3
G �G�	G�(G�'
G�Gc
��ttj�dk7r$tt�tj
d�t
tjdtjd�\}}td�|jj�D��}td|ddd�d|�d	t|j��d
��tjd|j�}tj�5|j|�}ddd�td|j�dtj ��d
��y#1swY�9xYw)Nr�rrr4c3�<K�|]}|j����y�wr.)�numel)r/r�s  rr2zmain.<locals>.<genexpr>)s����>�&=��1�7�7�9�&=�s�zloaded r�r�u — z params across z Linearsz	forward [u] → u  ✓)r��sys�argv�print�__doc__�exitr��sumr%�
parametersr;�randnr �no_gradr�rrv)r�r��n_paramsrO�ys     r�mainr�$s���
�3�8�8�}���
�g��������s�x�x��{�C�H�H�Q�K�8�J�C���>�c�j�j�&;�&;�&=�>�>�H�	�G�E�&�M�"�1�%�&�e�H�:�_�S����_�DU�U]�
^�_�
���A�s�}�}�%�A�	�����K�K��N��
�	�I�c�m�m�_�F�4����=�/��
?�@�
��s�>E�E�__main__r�)r�r�r��dataclassesrr�pathlibrr;�safetensors.torchrrrr�tupler�r�r�rrrr�<module>r�s����*�
�(���'�
�	#�	#��	#��{.�{.��{.�B-�3��:�-�C�-�E�'�4�-�<P�-�f
A��z���F�r