Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1158 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.4413. Min Act: -0.1700

Data Index: 7549824 (Open Web Text)

Max Activating Token Index: 312

Text #1

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.3199. Min Act: -0.1700

Data Index: 6218794 (Open Web Text)

Max Activating Token Index: 920

Text #2

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.2695. Min Act: -0.1700

Data Index: 1164179 (Open Web Text)

Max Activating Token Index: 828

Text #3

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.2744. Min Act: -0.1700

Data Index: 2361085 (Open Web Text)

Max Activating Token Index: 963

Text #4

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.1857. Min Act: -0.1700

Data Index: 455650 (Open Web Text)

Max Activating Token Index: 610

Text #5

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.1588. Min Act: -0.1700

Data Index: 6888558 (Open Web Text)

Max Activating Token Index: 543

Text #6

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.1012. Min Act: -0.1700

Data Index: 3935905 (Open Web Text)

Max Activating Token Index: 341

Text #7

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.1060. Min Act: -0.1700

Data Index: 3870183 (Open Web Text)

Max Activating Token Index: 63

Text #8

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.1035. Min Act: -0.1700

Data Index: 8104454 (Open Web Text)

Max Activating Token Index: 495

Text #9

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.0901. Min Act: -0.1700

Data Index: 7968387 (Open Web Text)

Max Activating Token Index: 483

Text #10

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.0095. Min Act: -0.1700

Data Index: 922800 (Open Web Text)

Max Activating Token Index: 85

Text #11

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.0535. Min Act: -0.1700

Data Index: 1040186 (Open Web Text)

Max Activating Token Index: 107

Text #12

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.0492. Min Act: -0.1700

Data Index: 1164180 (Open Web Text)

Max Activating Token Index: 905

Text #13

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.0099. Min Act: -0.1700

Data Index: 7199985 (Open Web Text)

Max Activating Token Index: 9

Text #14

Max Range: 5.4413. Min Range: -5.4413

Max Act: 5.0505. Min Act: -0.1700

Data Index: 1854746 (Open Web Text)

Max Activating Token Index: 1022

Text #15

Max Range: 5.4413. Min Range: -5.4413

Max Act: 4.9813. Min Act: -0.1700

Data Index: 5677341 (Open Web Text)

Max Activating Token Index: 675

Text #16

Max Range: 5.4413. Min Range: -5.4413

Max Act: 4.9585. Min Act: -0.1700

Data Index: 1721413 (Open Web Text)

Max Activating Token Index: 849

Text #17

Max Range: 5.4413. Min Range: -5.4413

Max Act: 4.9436. Min Act: -0.1700

Data Index: 2409645 (Open Web Text)

Max Activating Token Index: 187

Text #18

Max Range: 5.4413. Min Range: -5.4413

Max Act: 4.9521. Min Act: -0.1700

Data Index: 6927070 (Open Web Text)

Max Activating Token Index: 802

Text #19

Max Range: 5.4413. Min Range: -5.4413

Max Act: 4.8982. Min Act: -0.1700

Data Index: 5069998 (Open Web Text)

Max Activating Token Index: 183