Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2058 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.4883. Min Act: -0.1700

Data Index: 2015142 (Open Web Text)

Max Activating Token Index: 524

Text #1

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.4947. Min Act: -0.1700

Data Index: 2084051 (Open Web Text)

Max Activating Token Index: 603

Text #2

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.3374. Min Act: -0.1700

Data Index: 2063327 (Open Web Text)

Max Activating Token Index: 697

Text #3

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2972. Min Act: -0.1700

Data Index: 3677615 (Open Web Text)

Max Activating Token Index: 580

Text #4

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2634. Min Act: -0.1700

Data Index: 5370880 (Open Web Text)

Max Activating Token Index: 520

Text #5

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2427. Min Act: -0.1700

Data Index: 3009946 (Open Web Text)

Max Activating Token Index: 898

Text #6

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2701. Min Act: -0.1700

Data Index: 687039 (Open Web Text)

Max Activating Token Index: 341

Text #7

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2168. Min Act: -0.1700

Data Index: 7087659 (Open Web Text)

Max Activating Token Index: 638

Text #8

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2466. Min Act: -0.1700

Data Index: 8536008 (Open Web Text)

Max Activating Token Index: 490

Text #9

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2437. Min Act: -0.1700

Data Index: 110194 (Open Web Text)

Max Activating Token Index: 428

Text #10

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2245. Min Act: -0.1700

Data Index: 5365071 (Open Web Text)

Max Activating Token Index: 449

Text #11

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2437. Min Act: -0.1700

Data Index: 8581075 (Open Web Text)

Max Activating Token Index: 541

Text #12

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2314. Min Act: -0.1700

Data Index: 3877630 (Open Web Text)

Max Activating Token Index: 653

Text #13

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2064. Min Act: -0.1700

Data Index: 6882335 (Open Web Text)

Max Activating Token Index: 39

Text #14

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.1832. Min Act: -0.1700

Data Index: 8686983 (Open Web Text)

Max Activating Token Index: 370

Text #15

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.1807. Min Act: -0.1700

Data Index: 680761 (Open Web Text)

Max Activating Token Index: 604

Text #16

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2134. Min Act: -0.1700

Data Index: 5090107 (Open Web Text)

Max Activating Token Index: 833

Text #17

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.2307. Min Act: -0.1700

Data Index: 8252517 (Open Web Text)

Max Activating Token Index: 574

Text #18

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.1881. Min Act: -0.1700

Data Index: 2459153 (Open Web Text)

Max Activating Token Index: 210

Text #19

Max Range: 3.4947. Min Range: -3.4947

Max Act: 3.1981. Min Act: -0.1700

Data Index: 6922944 (Open Web Text)

Max Activating Token Index: 951