Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1398 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.4371. Min Act: -0.1700

Data Index: 5849019 (Open Web Text)

Max Activating Token Index: 20

Text #1

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.4257. Min Act: -0.1700

Data Index: 168486 (Open Web Text)

Max Activating Token Index: 42

Text #2

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.4109. Min Act: -0.1700

Data Index: 4735769 (Open Web Text)

Max Activating Token Index: 33

Text #3

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.4047. Min Act: -0.1700

Data Index: 8609543 (Open Web Text)

Max Activating Token Index: 38

Text #4

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3929. Min Act: -0.1700

Data Index: 4539700 (Open Web Text)

Max Activating Token Index: 71

Text #5

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3825. Min Act: -0.1700

Data Index: 768792 (Open Web Text)

Max Activating Token Index: 11

Text #6

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3741. Min Act: -0.1700

Data Index: 1773292 (Open Web Text)

Max Activating Token Index: 643

Text #7

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3536. Min Act: -0.1700

Data Index: 3307885 (Open Web Text)

Max Activating Token Index: 1023

Text #8

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3568. Min Act: -0.1700

Data Index: 3152584 (Open Web Text)

Max Activating Token Index: 63

Text #9

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3543. Min Act: -0.1700

Data Index: 5150136 (Open Web Text)

Max Activating Token Index: 14

Text #10

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3609. Min Act: -0.1700

Data Index: 1064369 (Open Web Text)

Max Activating Token Index: 26

Text #11

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3637. Min Act: -0.1700

Data Index: 314691 (Open Web Text)

Max Activating Token Index: 815

Text #12

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3458. Min Act: -0.1695

Data Index: 8289080 (Open Web Text)

Max Activating Token Index: 71

Text #13

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3542. Min Act: -0.1700

Data Index: 7427680 (Open Web Text)

Max Activating Token Index: 77

Text #14

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3508. Min Act: -0.1700

Data Index: 6773367 (Open Web Text)

Max Activating Token Index: 10

Text #15

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3396. Min Act: -0.1700

Data Index: 7097230 (Open Web Text)

Max Activating Token Index: 272

Text #16

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3424. Min Act: -0.1700

Data Index: 6817849 (Open Web Text)

Max Activating Token Index: 40

Text #17

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3405. Min Act: -0.1700

Data Index: 6437724 (Open Web Text)

Max Activating Token Index: 447

Text #18

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3272. Min Act: -0.1700

Data Index: 78979 (Open Web Text)

Max Activating Token Index: 51

Text #19

Max Range: 1.4371. Min Range: -1.4371

Max Act: 1.3335. Min Act: -0.1700

Data Index: 2654727 (Open Web Text)

Max Activating Token Index: 11