Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1754 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.2340. Min Act: -0.1700

Data Index: 4061190 (Open Web Text)

Max Activating Token Index: 335

Text #1

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.2007. Min Act: -0.1700

Data Index: 5023848 (Open Web Text)

Max Activating Token Index: 264

Text #2

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.1986. Min Act: -0.1700

Data Index: 891494 (Open Web Text)

Max Activating Token Index: 868

Text #3

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.1652. Min Act: -0.1700

Data Index: 2102459 (Open Web Text)

Max Activating Token Index: 180

Text #4

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.1798. Min Act: -0.1700

Data Index: 4207944 (Open Web Text)

Max Activating Token Index: 213

Text #5

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.1335. Min Act: -0.1700

Data Index: 3213623 (Open Web Text)

Max Activating Token Index: 58

Text #6

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0303. Min Act: -0.1700

Data Index: 760617 (Open Web Text)

Max Activating Token Index: 23

Text #7

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0450. Min Act: -0.1700

Data Index: 5546259 (Open Web Text)

Max Activating Token Index: 155

Text #8

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0884. Min Act: -0.1700

Data Index: 2242379 (Open Web Text)

Max Activating Token Index: 408

Text #9

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0436. Min Act: -0.1700

Data Index: 1289558 (Open Web Text)

Max Activating Token Index: 140

Text #10

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0266. Min Act: -0.1700

Data Index: 6884150 (Open Web Text)

Max Activating Token Index: 343

Text #11

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0473. Min Act: -0.1700

Data Index: 6928310 (Open Web Text)

Max Activating Token Index: 91

Text #12

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0474. Min Act: -0.1700

Data Index: 4003130 (Open Web Text)

Max Activating Token Index: 662

Text #13

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0238. Min Act: -0.1700

Data Index: 7487820 (Open Web Text)

Max Activating Token Index: 389

Text #14

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0344. Min Act: -0.1700

Data Index: 3291288 (Open Web Text)

Max Activating Token Index: 502

Text #15

Max Range: 3.2340. Min Range: -3.2340

Max Act: 2.9995. Min Act: -0.1700

Data Index: 6622090 (Open Web Text)

Max Activating Token Index: 184

Text #16

Max Range: 3.2340. Min Range: -3.2340

Max Act: 2.9926. Min Act: -0.1700

Data Index: 2116934 (Open Web Text)

Max Activating Token Index: 691

Text #17

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0227. Min Act: -0.1700

Data Index: 5630455 (Open Web Text)

Max Activating Token Index: 957

Text #18

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0151. Min Act: -0.1700

Data Index: 7789558 (Open Web Text)

Max Activating Token Index: 101

Text #19

Max Range: 3.2340. Min Range: -3.2340

Max Act: 3.0068. Min Act: -0.1700

Data Index: 4205067 (Open Web Text)

Max Activating Token Index: 715