Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2613 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.4371. Min Act: -0.1700

Data Index: 611701 (Open Web Text)

Max Activating Token Index: 550

Text #1

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.3895. Min Act: -0.1700

Data Index: 5801824 (Open Web Text)

Max Activating Token Index: 498

Text #2

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.3426. Min Act: -0.1700

Data Index: 2491338 (Open Web Text)

Max Activating Token Index: 407

Text #3

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.2992. Min Act: -0.1700

Data Index: 5159869 (Open Web Text)

Max Activating Token Index: 896

Text #4

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.3041. Min Act: -0.1700

Data Index: 2714923 (Open Web Text)

Max Activating Token Index: 154

Text #5

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.2674. Min Act: -0.1700

Data Index: 7611324 (Open Web Text)

Max Activating Token Index: 384

Text #6

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.2681. Min Act: -0.1700

Data Index: 2159092 (Open Web Text)

Max Activating Token Index: 323

Text #7

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.2438. Min Act: -0.1700

Data Index: 4488678 (Open Web Text)

Max Activating Token Index: 43

Text #8

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.2049. Min Act: -0.1700

Data Index: 2851414 (Open Web Text)

Max Activating Token Index: 748

Text #9

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.2084. Min Act: -0.1700

Data Index: 5930377 (Open Web Text)

Max Activating Token Index: 598

Text #10

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.2425. Min Act: -0.1700

Data Index: 3007872 (Open Web Text)

Max Activating Token Index: 896

Text #11

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.1902. Min Act: -0.1700

Data Index: 3653594 (Open Web Text)

Max Activating Token Index: 975

Text #12

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.1762. Min Act: -0.1700

Data Index: 3276480 (Open Web Text)

Max Activating Token Index: 186

Text #13

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.2067. Min Act: -0.1700

Data Index: 6519044 (Open Web Text)

Max Activating Token Index: 289

Text #14

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.1896. Min Act: -0.1700

Data Index: 7570658 (Open Web Text)

Max Activating Token Index: 939

Text #15

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.2002. Min Act: -0.1700

Data Index: 5357021 (Open Web Text)

Max Activating Token Index: 833

Text #16

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.1604. Min Act: -0.1700

Data Index: 4234183 (Open Web Text)

Max Activating Token Index: 344

Text #17

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.1747. Min Act: -0.1700

Data Index: 2110524 (Open Web Text)

Max Activating Token Index: 171

Text #18

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.1394. Min Act: -0.1700

Data Index: 5123073 (Open Web Text)

Max Activating Token Index: 860

Text #19

Max Range: 4.4371. Min Range: -4.4371

Max Act: 4.1343. Min Act: -0.1700

Data Index: 6314824 (Open Web Text)

Max Activating Token Index: 492