Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 413 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.4331. Min Act: -0.1700

Data Index: 2368051 (Open Web Text)

Max Activating Token Index: 557

Text #1

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.4125. Min Act: -0.1700

Data Index: 2722964 (Open Web Text)

Max Activating Token Index: 621

Text #2

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.3880. Min Act: -0.1700

Data Index: 7004666 (Open Web Text)

Max Activating Token Index: 633

Text #3

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.3155. Min Act: -0.1700

Data Index: 1783188 (Open Web Text)

Max Activating Token Index: 987

Text #4

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.3046. Min Act: -0.1700

Data Index: 1961238 (Open Web Text)

Max Activating Token Index: 884

Text #5

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2742. Min Act: -0.1700

Data Index: 5644718 (Open Web Text)

Max Activating Token Index: 304

Text #6

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2149. Min Act: -0.1700

Data Index: 3070196 (Open Web Text)

Max Activating Token Index: 739

Text #7

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2378. Min Act: -0.1700

Data Index: 3136821 (Open Web Text)

Max Activating Token Index: 634

Text #8

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2289. Min Act: -0.1700

Data Index: 180803 (Open Web Text)

Max Activating Token Index: 818

Text #9

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2436. Min Act: -0.1700

Data Index: 6686335 (Open Web Text)

Max Activating Token Index: 1019

Text #10

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2507. Min Act: -0.1700

Data Index: 1728627 (Open Web Text)

Max Activating Token Index: 883

Text #11

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2184. Min Act: -0.1700

Data Index: 2979437 (Open Web Text)

Max Activating Token Index: 255

Text #12

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.1937. Min Act: -0.1700

Data Index: 6459791 (Open Web Text)

Max Activating Token Index: 421

Text #13

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2434. Min Act: -0.1700

Data Index: 1651541 (Open Web Text)

Max Activating Token Index: 966

Text #14

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2089. Min Act: -0.1700

Data Index: 3973013 (Open Web Text)

Max Activating Token Index: 504

Text #15

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2090. Min Act: -0.1700

Data Index: 3902663 (Open Web Text)

Max Activating Token Index: 14

Text #16

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.1767. Min Act: -0.1700

Data Index: 647481 (Open Web Text)

Max Activating Token Index: 203

Text #17

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.1759. Min Act: -0.1700

Data Index: 5716036 (Open Web Text)

Max Activating Token Index: 590

Text #18

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.2069. Min Act: -0.1700

Data Index: 1791153 (Open Web Text)

Max Activating Token Index: 901

Text #19

Max Range: 4.4331. Min Range: -4.4331

Max Act: 4.1398. Min Act: -0.1700

Data Index: 7118796 (Open Web Text)

Max Activating Token Index: 911