Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2950 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.4187. Min Act: -0.1700

Data Index: 8165297 (Open Web Text)

Max Activating Token Index: 279

Text #1

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.2470. Min Act: -0.1700

Data Index: 1861376 (Open Web Text)

Max Activating Token Index: 165

Text #2

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.1864. Min Act: -0.1700

Data Index: 835530 (Open Web Text)

Max Activating Token Index: 693

Text #3

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.1434. Min Act: -0.1700

Data Index: 2017568 (Open Web Text)

Max Activating Token Index: 643

Text #4

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.1533. Min Act: -0.1700

Data Index: 3697769 (Open Web Text)

Max Activating Token Index: 409

Text #5

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.1061. Min Act: -0.1700

Data Index: 671111 (Open Web Text)

Max Activating Token Index: 841

Text #6

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.1004. Min Act: -0.1700

Data Index: 6226913 (Open Web Text)

Max Activating Token Index: 654

Text #7

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.1197. Min Act: -0.1700

Data Index: 4346042 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.0856. Min Act: -0.1700

Data Index: 7709068 (Open Web Text)

Max Activating Token Index: 636

Text #9

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.0995. Min Act: -0.1700

Data Index: 6226914 (Open Web Text)

Max Activating Token Index: 632

Text #10

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.1082. Min Act: -0.1700

Data Index: 7157257 (Open Web Text)

Max Activating Token Index: 671

Text #11

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.0692. Min Act: -0.1700

Data Index: 1335748 (Open Web Text)

Max Activating Token Index: 440

Text #12

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.0373. Min Act: -0.1700

Data Index: 123917 (Open Web Text)

Max Activating Token Index: 684

Text #13

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.0346. Min Act: -0.1700

Data Index: 7932899 (Open Web Text)

Max Activating Token Index: 221

Text #14

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.0126. Min Act: -0.1700

Data Index: 7565779 (Open Web Text)

Max Activating Token Index: 754

Text #15

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.0372. Min Act: -0.1700

Data Index: 8113621 (Open Web Text)

Max Activating Token Index: 254

Text #16

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.0570. Min Act: -0.1700

Data Index: 870402 (Open Web Text)

Max Activating Token Index: 703

Text #17

Max Range: 4.4187. Min Range: -4.4187

Max Act: 3.9915. Min Act: -0.1700

Data Index: 1360342 (Open Web Text)

Max Activating Token Index: 619

Text #18

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.0033. Min Act: -0.1700

Data Index: 1864563 (Open Web Text)

Max Activating Token Index: 395

Text #19

Max Range: 4.4187. Min Range: -4.4187

Max Act: 4.0193. Min Act: -0.1700

Data Index: 5758062 (Open Web Text)

Max Activating Token Index: 324