< Prev | Home | Model | Random | Next >

Model: SoLU Model: 6 Layers, 3072 Neurons per Layer

Dataset: The Pile

Neuron 3071 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-6l-pile')



Text #0

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0865. Min Act: -0.0001

Data Index: 1868678 (The Pile)

Max Activating Token Index: 576

Click toggle to see full text

Truncated

ა�ი� �ა�ე�ე�? 
>> ა� �ა�ა� �. 
ა�ა�ი�ე�. 
ა� �

Full Text #0

<|endoftext|>. 
>> � स �. 
 � � म �ि � ह 
 �, स क म � ह. 
 � म � ह. 
 � � स कि �, � � � ह 
 ह � ह प क क �ि � � �. 
 स क म 
 �िि ह. 
>> �ि कि � � ह. 
 � � ह. 
 म �िि � �. 
. 
, � � ह. 
 कि क क � 
 स � ह. 
 �ि स सि क क �ि � � � ह? 
? 
>> �: � � .c म � ह. 
>> � Hirschhorn: �ि स. 
 
Georgian:  
ი�, � � �ა�ა�ა�ე�. 
 �ე�ი� ა�ი� �ა�ე�ე�? 
>> ა� �ა�ა� �. 
ა�ა�ი�ე�. 
ა� �ე� � �ი�ა�ე�, ls, �ა�. 
cd �ი�ა� �ი�ე�ე�. 
cd dot dot bump, �ა�ე� �ი�ე�ე�. 
 ა� �ე� �ა�ი�ე� 
ი� �ა� �ა�ე� �ი�. 
ი� �ა�ა�ი�ე� �ა� �ა�ა�ი�ე� hi.txt � 
 �ა�ი�ე� �ა�ე� � �ა�ა�ა�. 
>> �ა� �ი�ა�. 
ე�ა� �ე� �ე� 
ა�ე�ი�, �ე�ა�ე� �ა�ა�. 
ა� �ე� �ა�ი�ე�. 
 � �ა�ე�, ა�ა�, � 
ა� ა�ი�ე�ე� �ი�, � �ე�. 
ა� �, �ე�ა�ე� �ა�ი�ე�

Text #1

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0857. Min Act: -0.0001

Data Index: 1755319 (The Pile)

Max Activating Token Index: 616

Click toggle to see full text

Truncated

Full Text #1


Text #2

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0783. Min Act: -0.0002

Data Index: 277391 (The Pile)

Max Activating Token Index: 777

Click toggle to see full text

Truncated

Full Text #2


Text #3

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0797. Min Act: -0.0002

Data Index: 133109 (The Pile)

Max Activating Token Index: 680

Click toggle to see full text

Truncated

Full Text #3


Text #4

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0763. Min Act: -0.0001

Data Index: 1089776 (The Pile)

Max Activating Token Index: 582

Click toggle to see full text

Truncated

Full Text #4


Text #5

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0756. Min Act: -0.0001

Data Index: 1465468 (The Pile)

Max Activating Token Index: 806

Click toggle to see full text

Truncated

Full Text #5


Text #6

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0741. Min Act: -0.0001

Data Index: 285520 (The Pile)

Max Activating Token Index: 980

Click toggle to see full text

Truncated

Full Text #6


Text #7

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0732. Min Act: -0.0001

Data Index: 711922 (The Pile)

Max Activating Token Index: 303

Click toggle to see full text

Truncated

Full Text #7


Text #8

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0733. Min Act: -0.0001

Data Index: 1784758 (The Pile)

Max Activating Token Index: 242

Click toggle to see full text

Truncated

Full Text #8


Text #9

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0743. Min Act: -0.0001

Data Index: 608631 (The Pile)

Max Activating Token Index: 411

Click toggle to see full text

Truncated

Full Text #9


Text #10

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0746. Min Act: -0.0001

Data Index: 1221699 (The Pile)

Max Activating Token Index: 90

Click toggle to see full text

Truncated

Full Text #10


Text #11

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0744. Min Act: -0.0001

Data Index: 725888 (The Pile)

Max Activating Token Index: 675

Click toggle to see full text

Truncated

Full Text #11


Text #12

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0710. Min Act: -0.0001

Data Index: 1787163 (The Pile)

Max Activating Token Index: 323

Click toggle to see full text

Truncated

Full Text #12


Text #13

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0708. Min Act: -0.0001

Data Index: 456279 (The Pile)

Max Activating Token Index: 564

Click toggle to see full text

Truncated

Full Text #13


Text #14

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0705. Min Act: -0.0001

Data Index: 1358600 (The Pile)

Max Activating Token Index: 221

Click toggle to see full text

Truncated

Full Text #14


Text #15

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0704. Min Act: -0.0002

Data Index: 194466 (The Pile)

Max Activating Token Index: 843

Click toggle to see full text

Truncated

Full Text #15


Text #16

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0698. Min Act: -0.0001

Data Index: 410129 (The Pile)

Max Activating Token Index: 246

Click toggle to see full text

Truncated

Full Text #16


Text #17

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0701. Min Act: -0.0001

Data Index: 1101394 (The Pile)

Max Activating Token Index: 575

Click toggle to see full text

Truncated

Full Text #17


Text #18

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0697. Min Act: -0.0001

Data Index: 774930 (The Pile)

Max Activating Token Index: 786

Click toggle to see full text

Truncated

Full Text #18


Text #19

Max Range: 0.0865. Min Range: -0.0865

Max Act: 0.0721. Min Act: -0.0001

Data Index: 244796 (The Pile)

Max Activating Token Index: 282

Click toggle to see full text

Truncated

Full Text #19