Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1028 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.8031. Min Act: -0.1700

Data Index: 2159128 (Open Web Text)

Max Activating Token Index: 610

Text #1

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.7218. Min Act: -0.1700

Data Index: 4472169 (Open Web Text)

Max Activating Token Index: 795

Text #2

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.7425. Min Act: -0.1700

Data Index: 3130497 (Open Web Text)

Max Activating Token Index: 942

Text #3

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.7072. Min Act: -0.1700

Data Index: 1814020 (Open Web Text)

Max Activating Token Index: 916

Text #4

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.7115. Min Act: -0.1700

Data Index: 3736178 (Open Web Text)

Max Activating Token Index: 659

Text #5

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6814. Min Act: -0.1700

Data Index: 5657672 (Open Web Text)

Max Activating Token Index: 940

Text #6

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.7163. Min Act: -0.1700

Data Index: 2455043 (Open Web Text)

Max Activating Token Index: 706

Text #7

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6736. Min Act: -0.1700

Data Index: 5073038 (Open Web Text)

Max Activating Token Index: 670

Text #8

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6600. Min Act: -0.1700

Data Index: 324354 (Open Web Text)

Max Activating Token Index: 164

Text #9

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6693. Min Act: -0.1700

Data Index: 4675276 (Open Web Text)

Max Activating Token Index: 529

Text #10

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6855. Min Act: -0.1700

Data Index: 1223938 (Open Web Text)

Max Activating Token Index: 870

Text #11

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6680. Min Act: -0.1700

Data Index: 1571390 (Open Web Text)

Max Activating Token Index: 894

Text #12

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6167. Min Act: -0.1700

Data Index: 2567487 (Open Web Text)

Max Activating Token Index: 924

Text #13

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6336. Min Act: -0.1700

Data Index: 4786018 (Open Web Text)

Max Activating Token Index: 459

Text #14

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6511. Min Act: -0.1700

Data Index: 5736313 (Open Web Text)

Max Activating Token Index: 620

Text #15

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6190. Min Act: -0.1700

Data Index: 8303921 (Open Web Text)

Max Activating Token Index: 898

Text #16

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6215. Min Act: -0.1700

Data Index: 2970108 (Open Web Text)

Max Activating Token Index: 766

Text #17

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.5941. Min Act: -0.1700

Data Index: 3628773 (Open Web Text)

Max Activating Token Index: 370

Text #18

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.6219. Min Act: -0.1700

Data Index: 4343825 (Open Web Text)

Max Activating Token Index: 321

Text #19

Max Range: 2.8031. Min Range: -2.8031

Max Act: 2.5629. Min Act: -0.1700

Data Index: 1250687 (Open Web Text)

Max Activating Token Index: 467