Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1177 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.5856. Min Act: -0.1177

Data Index: 2903754 (Open Web Text)

Max Activating Token Index: 85

Text #1

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.5289. Min Act: -0.1108

Data Index: 1468178 (Open Web Text)

Max Activating Token Index: 393

Text #2

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.5009. Min Act: -0.1033

Data Index: 5289307 (Open Web Text)

Max Activating Token Index: 377

Text #3

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4914. Min Act: -0.1031

Data Index: 3724609 (Open Web Text)

Max Activating Token Index: 484

Text #4

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4800. Min Act: -0.1333

Data Index: 8440080 (Open Web Text)

Max Activating Token Index: 383

Text #5

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4740. Min Act: -0.1074

Data Index: 3328185 (Open Web Text)

Max Activating Token Index: 476

Text #6

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4510. Min Act: -0.1013

Data Index: 8599056 (Open Web Text)

Max Activating Token Index: 379

Text #7

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4560. Min Act: -0.0021

Data Index: 4248930 (Open Web Text)

Max Activating Token Index: 106

Text #8

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4441. Min Act: -0.0528

Data Index: 1396942 (Open Web Text)

Max Activating Token Index: 413

Text #9

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4443. Min Act: -0.1077

Data Index: 8575230 (Open Web Text)

Max Activating Token Index: 477

Text #10

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4498. Min Act: -0.0695

Data Index: 2444209 (Open Web Text)

Max Activating Token Index: 403

Text #11

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4396. Min Act: -0.1116

Data Index: 3005951 (Open Web Text)

Max Activating Token Index: 395

Text #12

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4379. Min Act: -0.0681

Data Index: 7105930 (Open Web Text)

Max Activating Token Index: 390

Text #13

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4350. Min Act: -0.0679

Data Index: 2211256 (Open Web Text)

Max Activating Token Index: 921

Text #14

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4405. Min Act: -0.1137

Data Index: 2441734 (Open Web Text)

Max Activating Token Index: 841

Text #15

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4372. Min Act: -0.1230

Data Index: 2211249 (Open Web Text)

Max Activating Token Index: 476

Text #16

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4438. Min Act: -0.0685

Data Index: 6396024 (Open Web Text)

Max Activating Token Index: 860

Text #17

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4407. Min Act: -0.1004

Data Index: 6396014 (Open Web Text)

Max Activating Token Index: 621

Text #18

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4274. Min Act: -0.0762

Data Index: 2029311 (Open Web Text)

Max Activating Token Index: 814

Text #19

Max Range: 1.5856. Min Range: -1.5856

Max Act: 1.4324. Min Act: 0.0439

Data Index: 3405257 (Open Web Text)

Max Activating Token Index: 88