Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2986 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4854. Min Act: -0.1700

Data Index: 5504548 (Open Web Text)

Max Activating Token Index: 350

Text #1

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4717. Min Act: -0.1700

Data Index: 908354 (Open Web Text)

Max Activating Token Index: 357

Text #2

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4658. Min Act: -0.1700

Data Index: 7733868 (Open Web Text)

Max Activating Token Index: 420

Text #3

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4519. Min Act: -0.1700

Data Index: 678723 (Open Web Text)

Max Activating Token Index: 330

Text #4

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4342. Min Act: -0.1700

Data Index: 593271 (Open Web Text)

Max Activating Token Index: 691

Text #5

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4508. Min Act: -0.1700

Data Index: 6142886 (Open Web Text)

Max Activating Token Index: 255

Text #6

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4319. Min Act: -0.1700

Data Index: 1442569 (Open Web Text)

Max Activating Token Index: 319

Text #7

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4342. Min Act: -0.1700

Data Index: 4410385 (Open Web Text)

Max Activating Token Index: 299

Text #8

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4310. Min Act: -0.1700

Data Index: 6795594 (Open Web Text)

Max Activating Token Index: 422

Text #9

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4314. Min Act: -0.1700

Data Index: 6662736 (Open Web Text)

Max Activating Token Index: 343

Text #10

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4312. Min Act: -0.1700

Data Index: 5768357 (Open Web Text)

Max Activating Token Index: 286

Text #11

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4302. Min Act: -0.1700

Data Index: 1773846 (Open Web Text)

Max Activating Token Index: 336

Text #12

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4322. Min Act: -0.1700

Data Index: 5302686 (Open Web Text)

Max Activating Token Index: 284

Text #13

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4311. Min Act: -0.1700

Data Index: 8439505 (Open Web Text)

Max Activating Token Index: 759

Text #14

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4296. Min Act: -0.1700

Data Index: 7797804 (Open Web Text)

Max Activating Token Index: 257

Text #15

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4192. Min Act: -0.1700

Data Index: 3353684 (Open Web Text)

Max Activating Token Index: 312

Text #16

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4014. Min Act: -0.1700

Data Index: 6601970 (Open Web Text)

Max Activating Token Index: 480

Text #17

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4114. Min Act: -0.1700

Data Index: 4117000 (Open Web Text)

Max Activating Token Index: 343

Text #18

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4065. Min Act: -0.1700

Data Index: 5297630 (Open Web Text)

Max Activating Token Index: 375

Text #19

Max Range: 2.4854. Min Range: -2.4854

Max Act: 2.4154. Min Act: -0.1700

Data Index: 5236046 (Open Web Text)

Max Activating Token Index: 330