Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 919 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.5664. Min Act: -0.1700

Data Index: 1119830 (Open Web Text)

Max Activating Token Index: 136

Text #1

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.5585. Min Act: -0.1700

Data Index: 2666119 (Open Web Text)

Max Activating Token Index: 376

Text #2

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.5424. Min Act: -0.1700

Data Index: 8730737 (Open Web Text)

Max Activating Token Index: 286

Text #3

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.5637. Min Act: -0.1700

Data Index: 4476304 (Open Web Text)

Max Activating Token Index: 304

Text #4

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4921. Min Act: -0.1700

Data Index: 7842282 (Open Web Text)

Max Activating Token Index: 548

Text #5

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.5200. Min Act: -0.1700

Data Index: 7073158 (Open Web Text)

Max Activating Token Index: 655

Text #6

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4660. Min Act: -0.1700

Data Index: 8368718 (Open Web Text)

Max Activating Token Index: 726

Text #7

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4895. Min Act: -0.1700

Data Index: 1168040 (Open Web Text)

Max Activating Token Index: 538

Text #8

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4908. Min Act: -0.1700

Data Index: 6802298 (Open Web Text)

Max Activating Token Index: 582

Text #9

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4347. Min Act: -0.1700

Data Index: 1211147 (Open Web Text)

Max Activating Token Index: 632

Text #10

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4464. Min Act: -0.1700

Data Index: 7613253 (Open Web Text)

Max Activating Token Index: 473

Text #11

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4296. Min Act: -0.1700

Data Index: 6153461 (Open Web Text)

Max Activating Token Index: 517

Text #12

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4412. Min Act: -0.1700

Data Index: 7326744 (Open Web Text)

Max Activating Token Index: 462

Text #13

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4691. Min Act: -0.1700

Data Index: 6910313 (Open Web Text)

Max Activating Token Index: 949

Text #14

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4384. Min Act: -0.1700

Data Index: 4336954 (Open Web Text)

Max Activating Token Index: 760

Text #15

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4479. Min Act: -0.1700

Data Index: 3399314 (Open Web Text)

Max Activating Token Index: 339

Text #16

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4504. Min Act: -0.1700

Data Index: 8669404 (Open Web Text)

Max Activating Token Index: 594

Text #17

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4366. Min Act: -0.1700

Data Index: 2961091 (Open Web Text)

Max Activating Token Index: 472

Text #18

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4329. Min Act: -0.1700

Data Index: 7278294 (Open Web Text)

Max Activating Token Index: 585

Text #19

Max Range: 4.5664. Min Range: -4.5664

Max Act: 4.4274. Min Act: -0.1700

Data Index: 6562075 (Open Web Text)

Max Activating Token Index: 306