Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2514 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.4250. Min Act: -0.1700

Data Index: 5645489 (Open Web Text)

Max Activating Token Index: 465

Text #1

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3698. Min Act: -0.1700

Data Index: 1561011 (Open Web Text)

Max Activating Token Index: 319

Text #2

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3399. Min Act: -0.1700

Data Index: 2991393 (Open Web Text)

Max Activating Token Index: 366

Text #3

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3246. Min Act: -0.1700

Data Index: 5959649 (Open Web Text)

Max Activating Token Index: 332

Text #4

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3137. Min Act: -0.1700

Data Index: 2587333 (Open Web Text)

Max Activating Token Index: 305

Text #5

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3135. Min Act: -0.1700

Data Index: 5807439 (Open Web Text)

Max Activating Token Index: 488

Text #6

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3028. Min Act: -0.1700

Data Index: 911948 (Open Web Text)

Max Activating Token Index: 370

Text #7

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3224. Min Act: -0.1700

Data Index: 1268787 (Open Web Text)

Max Activating Token Index: 504

Text #8

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3210. Min Act: -0.1700

Data Index: 7739940 (Open Web Text)

Max Activating Token Index: 651

Text #9

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3183. Min Act: -0.1700

Data Index: 122388 (Open Web Text)

Max Activating Token Index: 463

Text #10

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3114. Min Act: -0.1700

Data Index: 4905006 (Open Web Text)

Max Activating Token Index: 463

Text #11

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3101. Min Act: -0.1700

Data Index: 2058881 (Open Web Text)

Max Activating Token Index: 670

Text #12

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.3062. Min Act: -0.1700

Data Index: 3290391 (Open Web Text)

Max Activating Token Index: 412

Text #13

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.2998. Min Act: -0.1700

Data Index: 8627309 (Open Web Text)

Max Activating Token Index: 201

Text #14

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.2968. Min Act: -0.1700

Data Index: 2187169 (Open Web Text)

Max Activating Token Index: 211

Text #15

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.2987. Min Act: -0.1700

Data Index: 482745 (Open Web Text)

Max Activating Token Index: 707

Text #16

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.2991. Min Act: -0.1700

Data Index: 3813855 (Open Web Text)

Max Activating Token Index: 610

Text #17

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.2907. Min Act: -0.1700

Data Index: 47086 (Open Web Text)

Max Activating Token Index: 532

Text #18

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.2655. Min Act: -0.1700

Data Index: 3466968 (Open Web Text)

Max Activating Token Index: 235

Text #19

Max Range: 3.4250. Min Range: -3.4250

Max Act: 3.2754. Min Act: -0.1700

Data Index: 3555709 (Open Web Text)

Max Activating Token Index: 528