Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1757 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.5937. Min Act: -0.1700

Data Index: 1230166 (Open Web Text)

Max Activating Token Index: 842

Text #1

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.5484. Min Act: -0.1700

Data Index: 8514306 (Open Web Text)

Max Activating Token Index: 619

Text #2

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.5336. Min Act: -0.1700

Data Index: 8565098 (Open Web Text)

Max Activating Token Index: 386

Text #3

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.5071. Min Act: -0.1700

Data Index: 4732602 (Open Web Text)

Max Activating Token Index: 627

Text #4

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4938. Min Act: -0.1700

Data Index: 7534482 (Open Web Text)

Max Activating Token Index: 549

Text #5

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4773. Min Act: -0.1700

Data Index: 636751 (Open Web Text)

Max Activating Token Index: 990

Text #6

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4717. Min Act: -0.1700

Data Index: 402246 (Open Web Text)

Max Activating Token Index: 348

Text #7

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4678. Min Act: -0.1700

Data Index: 1230167 (Open Web Text)

Max Activating Token Index: 679

Text #8

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4398. Min Act: -0.1700

Data Index: 4003530 (Open Web Text)

Max Activating Token Index: 648

Text #9

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4575. Min Act: -0.1700

Data Index: 1967638 (Open Web Text)

Max Activating Token Index: 731

Text #10

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4618. Min Act: -0.1700

Data Index: 8716511 (Open Web Text)

Max Activating Token Index: 839

Text #11

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4332. Min Act: -0.1700

Data Index: 197923 (Open Web Text)

Max Activating Token Index: 169

Text #12

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4376. Min Act: -0.1700

Data Index: 53438 (Open Web Text)

Max Activating Token Index: 290

Text #13

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4232. Min Act: -0.1700

Data Index: 6091289 (Open Web Text)

Max Activating Token Index: 208

Text #14

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4326. Min Act: -0.1700

Data Index: 2555057 (Open Web Text)

Max Activating Token Index: 303

Text #15

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4464. Min Act: -0.1700

Data Index: 1929873 (Open Web Text)

Max Activating Token Index: 1008

Text #16

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4458. Min Act: -0.1700

Data Index: 1411507 (Open Web Text)

Max Activating Token Index: 429

Text #17

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4259. Min Act: -0.1700

Data Index: 695919 (Open Web Text)

Max Activating Token Index: 439

Text #18

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.4189. Min Act: -0.1700

Data Index: 196717 (Open Web Text)

Max Activating Token Index: 461

Text #19

Max Range: 4.5937. Min Range: -4.5937

Max Act: 4.3997. Min Act: -0.1700

Data Index: 1785046 (Open Web Text)

Max Activating Token Index: 668