Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2945 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.5270. Min Act: -0.1700

Data Index: 7626199 (Open Web Text)

Max Activating Token Index: 911

Text #1

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.4983. Min Act: -0.1700

Data Index: 4838498 (Open Web Text)

Max Activating Token Index: 45

Text #2

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.4631. Min Act: -0.1700

Data Index: 6406027 (Open Web Text)

Max Activating Token Index: 564

Text #3

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.4087. Min Act: -0.1700

Data Index: 5487136 (Open Web Text)

Max Activating Token Index: 938

Text #4

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3960. Min Act: -0.1700

Data Index: 1935319 (Open Web Text)

Max Activating Token Index: 915

Text #5

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.4090. Min Act: -0.1700

Data Index: 727747 (Open Web Text)

Max Activating Token Index: 647

Text #6

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3691. Min Act: -0.1700

Data Index: 7087690 (Open Web Text)

Max Activating Token Index: 13

Text #7

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3615. Min Act: -0.1700

Data Index: 4375685 (Open Web Text)

Max Activating Token Index: 70

Text #8

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3730. Min Act: -0.1700

Data Index: 1040245 (Open Web Text)

Max Activating Token Index: 126

Text #9

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3948. Min Act: -0.1700

Data Index: 8599580 (Open Web Text)

Max Activating Token Index: 754

Text #10

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3588. Min Act: -0.1700

Data Index: 6166083 (Open Web Text)

Max Activating Token Index: 405

Text #11

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3425. Min Act: -0.1700

Data Index: 2048879 (Open Web Text)

Max Activating Token Index: 382

Text #12

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3530. Min Act: -0.1700

Data Index: 1455309 (Open Web Text)

Max Activating Token Index: 685

Text #13

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3409. Min Act: -0.1700

Data Index: 7470377 (Open Web Text)

Max Activating Token Index: 118

Text #14

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3389. Min Act: -0.1700

Data Index: 2678038 (Open Web Text)

Max Activating Token Index: 277

Text #15

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3250. Min Act: -0.1700

Data Index: 6430358 (Open Web Text)

Max Activating Token Index: 519

Text #16

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3038. Min Act: -0.1700

Data Index: 6191167 (Open Web Text)

Max Activating Token Index: 647

Text #17

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3190. Min Act: -0.1700

Data Index: 767966 (Open Web Text)

Max Activating Token Index: 496

Text #18

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.3286. Min Act: -0.1700

Data Index: 6254266 (Open Web Text)

Max Activating Token Index: 705

Text #19

Max Range: 5.5270. Min Range: -5.5270

Max Act: 5.2900. Min Act: -0.1700

Data Index: 8154247 (Open Web Text)

Max Activating Token Index: 888