Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1935 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.5075. Min Act: -0.1700

Data Index: 5974382 (Open Web Text)

Max Activating Token Index: 383

Text #1

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.4695. Min Act: -0.1700

Data Index: 7832135 (Open Web Text)

Max Activating Token Index: 1023

Text #2

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.4204. Min Act: -0.1700

Data Index: 3767535 (Open Web Text)

Max Activating Token Index: 172

Text #3

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.3793. Min Act: -0.1700

Data Index: 6458448 (Open Web Text)

Max Activating Token Index: 260

Text #4

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.3540. Min Act: -0.1700

Data Index: 685371 (Open Web Text)

Max Activating Token Index: 167

Text #5

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.3786. Min Act: -0.1700

Data Index: 8212469 (Open Web Text)

Max Activating Token Index: 206

Text #6

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.3758. Min Act: -0.1700

Data Index: 7377847 (Open Web Text)

Max Activating Token Index: 128

Text #7

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.3263. Min Act: -0.1700

Data Index: 1245804 (Open Web Text)

Max Activating Token Index: 913

Text #8

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.3059. Min Act: -0.1700

Data Index: 2987100 (Open Web Text)

Max Activating Token Index: 828

Text #9

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.3352. Min Act: -0.1700

Data Index: 3251692 (Open Web Text)

Max Activating Token Index: 495

Text #10

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.3255. Min Act: -0.1700

Data Index: 6876481 (Open Web Text)

Max Activating Token Index: 433

Text #11

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.2991. Min Act: -0.1700

Data Index: 4199209 (Open Web Text)

Max Activating Token Index: 723

Text #12

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.2919. Min Act: -0.1700

Data Index: 3581333 (Open Web Text)

Max Activating Token Index: 490

Text #13

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.2552. Min Act: -0.1700

Data Index: 8577937 (Open Web Text)

Max Activating Token Index: 189

Text #14

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.2667. Min Act: -0.1700

Data Index: 1754177 (Open Web Text)

Max Activating Token Index: 471

Text #15

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.2346. Min Act: -0.1700

Data Index: 6272419 (Open Web Text)

Max Activating Token Index: 838

Text #16

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.2175. Min Act: -0.1700

Data Index: 4488629 (Open Web Text)

Max Activating Token Index: 530

Text #17

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.2305. Min Act: -0.1700

Data Index: 2190784 (Open Web Text)

Max Activating Token Index: 818

Text #18

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.2284. Min Act: -0.1700

Data Index: 3712213 (Open Web Text)

Max Activating Token Index: 941

Text #19

Max Range: 4.5075. Min Range: -4.5075

Max Act: 4.2205. Min Act: -0.1700

Data Index: 8418256 (Open Web Text)

Max Activating Token Index: 281