B
��^� � @ s� d dl Zd dlm mZ d dlZd dlZd dlm Z d dl
Z
d dl m Z d dl
mZ d dlmZ d dlmZ d dlZd dlmZ G dd � d �ZdS )
� N)�Path)�get_from_cache)�
CTRLTokenizer)�PyCtrlTokenizer)� CTRLModel)�
default_timerc @ sL e Zd Zdd� Zdd� Zdd� Zdd� Zd d
� Zd d � Zd
d� Z dd� Z
dS )�TestBenchmarkCTRLc s t j�� � _tt�� �� _tj dd� jd�� _
t t � j
j
d d �t � j
j
d d ��� _tj ddd��� � _� jr~� j�� dgd � _� fd
d �� jD �}� fd d �|D �}� fd
d �|D �}t jdd � |D �t jd�}� jr�|�� }t �� � � �|�d �� �� }W d Q R X d S )N�ctrlT)�
do_lower_case� cache_dir�
vocab_file� merges_fileF)�output_attentionsu� For instance, on the planet Earth, man had always assumed that he was more intelligent than dolphins because he had achieved so much—the wheel, New York, wars and so on—whilst all the dolphins had ever done was muck about in the water having a good time. But conversely, the dolphins had always believed that they were far more intelligent than man—for precisely the same reasons.� c s g | ]}� j �|��qS � )�base_tokenizer�tokenize)�.0�sentence)�selfr �DE:\Coding\backup-rust\rust-transformers\tests\test_benchmark_ctrl.py�
<listcomp>. s z1TestBenchmarkCTRL.setup_class.<locals>.<listcomp>c s g | ]}� j �|��qS r )r �convert_tokens_to_ids)r �tokens)r r r r / s c s g | ]}� j j|d ddd��qS )NT� )�add_special_tokens�
max_length)r �prepare_for_model)r �input)r r r r 0 s c S s g | ] }|d �qS )� input_idsr )r �fr r r r 3 s )�dtyper )�torch�cuda� is_available�use_gpur �tempfile�mkdtemp�test_dirr �from_pretrainedr r r �pretrained_vocab_files_map�rust_tokenizerr �eval�model�
sentence_list�tensor�long�no_grad�cpu�numpy)r � tokens_list�features�
all_input_ids�_r )r r � setup_class s,
z TestBenchmarkCTRL.setup_classc C s t jdd| jd�| _d S )Nr T)r
r )r r) r( r )r r r r �setup_base_tokenizer; s z&TestBenchmarkCTRL.setup_base_tokenizerc C s0 t t| jjd d �t| jjd d ��| _d S )Nr r r
)r r r r* r+ )r r r r �setup_rust_tokenizer? s z&TestBenchmarkCTRL.setup_rust_tokenizerc s� � fdd�� j D �}� fdd�|D �}� fdd�|D �}tjdd� |D �tjd�}� jr`|�� }t�� � � �|�d �� � � }W d Q R X |S )Nc s g | ]}� j �|��qS r )r r )r r )r r r r F s z4TestBenchmarkCTRL.baseline_batch.<locals>.<listcomp>c s g | ]}� j �|��qS r )r r )r r )r r r r G s c s g | ]}� j j|d ddd��qS )NTr )r r )r r )r r )r r r r H s c S s g | ] }|d �qS )r r )r r r r r r L s )r! r )
r. r" r/ r0 r% r# r1 r- r2 r3 )r r4 r5 r6 �outputr )r r �baseline_batchE s
z TestBenchmarkCTRL.baseline_batchc sj � fdd�� j D �}tjdd� |D �tjd�}� jr<|�� }t�� � � �|�d �� � � }W d Q R X |S )Nc s g | ]}� j j|d ddd��qS )r �
longest_firstr )�max_len�truncation_strategy�stride)r+ �encode)r r )r r r r T s z@TestBenchmarkCTRL.rust_batch_single_threaded.<locals>.<listcomp>c S s g | ]
}|j �qS r )� token_ids)r r r r r r X s )r! r )
r. r" r/ r0 r% r# r1 r- r2 r3 )r r5 r6 r; r )r r �rust_batch_single_threadedS s
z,TestBenchmarkCTRL.rust_batch_single_threadedc s� g }x>t d�D ]2}| �� t� }| �� t� }|�|| d � qW t|�t|� � t�t� fdd�|D ���t|�d }t d� d�d|d��� d S ) N�
i� c s g | ]}|� d �qS )� r )r �value)�meanr r r h s z8TestBenchmarkCTRL.test_ctrl_baseline.<locals>.<listcomp>r zbaseline - mean: z.2fz , std. dev: )
�ranger9 �timerr< �append�sum�len�math�sqrt�print)r �values�i�t0�t1�std_devr )rG r �test_ctrl_baseline_ s (z$TestBenchmarkCTRL.test_ctrl_baselinec s� g }x>t d�D ]2}| �� t� }| �� t� }|�|| d � qW t|�t|� � t�t� fdd�|D ���t|�d }t d� d�d|d��� d S ) NrD i� c s g | ]}|� d �qS )rE r )r rF )rG r r r t s zDTestBenchmarkCTRL.test_ctrl_rust_single_threaded.<locals>.<listcomp>r zrust single thread - mean: z.2fz , std. dev: )
rH r: rI rC rJ rK rL rM rN rO )r rP rQ rR rS rT r )rG r � test_ctrl_rust_single_threadedk s (z0TestBenchmarkCTRL.test_ctrl_rust_single_threadedc C s( d | _ d | _d | _t�� tj�� d S )N)r- r r+ �gc�collectr" r# � empty_cache)r r r r �teardown_classw s
z TestBenchmarkCTRL.teardown_classN) �__name__�
__module__� __qualname__r8 r9 r: r< rC rU rV rZ r r r r r s " r )�builtins� @py_builtins�_pytest.assertion.rewrite� assertion�rewrite�
@pytest_arrM r& �pathlibr rW Ztransformers.file_utilsr Z transformers.tokenization_ctrlr �rust_transformersr Ztransformers.modeling_ctrlr r" �timeitr rI r r r r r �<module> s