Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2533 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.4325. Min Act: -0.1700

Data Index: 1771129 (Open Web Text)

Max Activating Token Index: 88

Text #1

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.3206. Min Act: -0.1700

Data Index: 1041439 (Open Web Text)

Max Activating Token Index: 363

Text #2

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.3011. Min Act: -0.1700

Data Index: 3322179 (Open Web Text)

Max Activating Token Index: 893

Text #3

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.3236. Min Act: -0.1700

Data Index: 1787947 (Open Web Text)

Max Activating Token Index: 239

Text #4

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.3438. Min Act: -0.1700

Data Index: 7454814 (Open Web Text)

Max Activating Token Index: 634

Text #5

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2788. Min Act: -0.1700

Data Index: 5927733 (Open Web Text)

Max Activating Token Index: 829

Text #6

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2896. Min Act: -0.1700

Data Index: 7294230 (Open Web Text)

Max Activating Token Index: 794

Text #7

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2808. Min Act: -0.1700

Data Index: 5699102 (Open Web Text)

Max Activating Token Index: 440

Text #8

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2694. Min Act: -0.1700

Data Index: 5461738 (Open Web Text)

Max Activating Token Index: 306

Text #9

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2627. Min Act: -0.1700

Data Index: 2056079 (Open Web Text)

Max Activating Token Index: 969

Text #10

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2688. Min Act: -0.1700

Data Index: 3823763 (Open Web Text)

Max Activating Token Index: 873

Text #11

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2619. Min Act: -0.1700

Data Index: 6534071 (Open Web Text)

Max Activating Token Index: 47

Text #12

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2928. Min Act: -0.1700

Data Index: 4135457 (Open Web Text)

Max Activating Token Index: 295

Text #13

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2279. Min Act: -0.1700

Data Index: 666686 (Open Web Text)

Max Activating Token Index: 415

Text #14

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2709. Min Act: -0.1700

Data Index: 5136184 (Open Web Text)

Max Activating Token Index: 478

Text #15

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2277. Min Act: -0.1700

Data Index: 5715871 (Open Web Text)

Max Activating Token Index: 62

Text #16

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2294. Min Act: -0.1700

Data Index: 5661296 (Open Web Text)

Max Activating Token Index: 100

Text #17

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2437. Min Act: -0.1700

Data Index: 5420828 (Open Web Text)

Max Activating Token Index: 163

Text #18

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2222. Min Act: -0.1700

Data Index: 4719953 (Open Web Text)

Max Activating Token Index: 566

Text #19

Max Range: 4.4325. Min Range: -4.4325

Max Act: 4.2384. Min Act: -0.1700

Data Index: 6741800 (Open Web Text)

Max Activating Token Index: 148