Model: GELU Model: 1 Layers, 2048 Neurons per Layer

Dataset: 80% C4 (Web Text) and 20% Python Code

Neuron 1266 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gelu-1l')

Click the toggle to see the full text!

Text #0

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.6490. Min Act: -0.1700

Data Index: 1115665 (C4 (Web Text))

Max Activating Token Index: 657

Text #1

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.5850. Min Act: -0.1700

Data Index: 1194141 (C4 (Web Text))

Max Activating Token Index: 531

Text #2

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.5810. Min Act: -0.1700

Data Index: 252537 (C4 (Web Text))

Max Activating Token Index: 474

Text #3

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.5135. Min Act: -0.1700

Data Index: 343476 (C4 (Web Text))

Max Activating Token Index: 220

Text #4

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.5334. Min Act: -0.1700

Data Index: 90185 (C4 (Web Text))

Max Activating Token Index: 100

Text #5

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.5128. Min Act: -0.1700

Data Index: 1173565 (C4 (Web Text))

Max Activating Token Index: 800

Text #6

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.5223. Min Act: -0.1700

Data Index: 49668 (C4 (Web Text))

Max Activating Token Index: 674

Text #7

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4936. Min Act: -0.1700

Data Index: 766627 (C4 (Web Text))

Max Activating Token Index: 395

Text #8

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4606. Min Act: -0.1700

Data Index: 901585 (C4 (Web Text))

Max Activating Token Index: 211

Text #9

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4499. Min Act: -0.1700

Data Index: 233431 (C4 (Web Text))

Max Activating Token Index: 167

Text #10

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4324. Min Act: -0.1700

Data Index: 942798 (C4 (Web Text))

Max Activating Token Index: 168

Text #11

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4567. Min Act: -0.1700

Data Index: 1307486 (C4 (Web Text))

Max Activating Token Index: 572

Text #12

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4276. Min Act: -0.1700

Data Index: 1320896 (C4 (Web Text))

Max Activating Token Index: 956

Text #13

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4476. Min Act: -0.1700

Data Index: 1023415 (C4 (Web Text))

Max Activating Token Index: 379

Text #14

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4572. Min Act: -0.1700

Data Index: 1058842 (C4 (Web Text))

Max Activating Token Index: 834

Text #15

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4404. Min Act: -0.1700

Data Index: 799927 (C4 (Web Text))

Max Activating Token Index: 708

Text #16

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4313. Min Act: -0.1700

Data Index: 1192004 (C4 (Web Text))

Max Activating Token Index: 660

Text #17

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4177. Min Act: -0.1700

Data Index: 1053346 (C4 (Web Text))

Max Activating Token Index: 253

Text #18

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.4058. Min Act: -0.1700

Data Index: 915343 (C4 (Web Text))

Max Activating Token Index: 22

Text #19

Max Range: 4.6490. Min Range: -4.6490

Max Act: 4.3809. Min Act: -0.1700

Data Index: 69513 (C4 (Web Text))

Max Activating Token Index: 815