Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2710 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.3345. Min Act: -0.1700

Data Index: 7552310 (Open Web Text)

Max Activating Token Index: 765

Text #1

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.3180. Min Act: -0.1700

Data Index: 102144 (Open Web Text)

Max Activating Token Index: 65

Text #2

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.3205. Min Act: -0.1700

Data Index: 7785061 (Open Web Text)

Max Activating Token Index: 675

Text #3

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2970. Min Act: -0.1700

Data Index: 320621 (Open Web Text)

Max Activating Token Index: 223

Text #4

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2677. Min Act: -0.1700

Data Index: 2001242 (Open Web Text)

Max Activating Token Index: 192

Text #5

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2695. Min Act: -0.1700

Data Index: 386946 (Open Web Text)

Max Activating Token Index: 379

Text #6

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2418. Min Act: -0.1700

Data Index: 2417512 (Open Web Text)

Max Activating Token Index: 214

Text #7

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2540. Min Act: -0.1700

Data Index: 1507982 (Open Web Text)

Max Activating Token Index: 870

Text #8

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2457. Min Act: -0.1700

Data Index: 7553319 (Open Web Text)

Max Activating Token Index: 615

Text #9

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2706. Min Act: -0.1700

Data Index: 4024990 (Open Web Text)

Max Activating Token Index: 679

Text #10

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2450. Min Act: -0.1700

Data Index: 3697349 (Open Web Text)

Max Activating Token Index: 935

Text #11

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2016. Min Act: -0.1700

Data Index: 1244915 (Open Web Text)

Max Activating Token Index: 315

Text #12

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2079. Min Act: -0.1700

Data Index: 596479 (Open Web Text)

Max Activating Token Index: 482

Text #13

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2219. Min Act: -0.1700

Data Index: 2793789 (Open Web Text)

Max Activating Token Index: 344

Text #14

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2484. Min Act: -0.1700

Data Index: 3518894 (Open Web Text)

Max Activating Token Index: 632

Text #15

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2183. Min Act: -0.1700

Data Index: 3215720 (Open Web Text)

Max Activating Token Index: 209

Text #16

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2064. Min Act: -0.1700

Data Index: 6311201 (Open Web Text)

Max Activating Token Index: 54

Text #17

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2145. Min Act: -0.1700

Data Index: 2657936 (Open Web Text)

Max Activating Token Index: 679

Text #18

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2069. Min Act: -0.1700

Data Index: 6126796 (Open Web Text)

Max Activating Token Index: 511

Text #19

Max Range: 5.3345. Min Range: -5.3345

Max Act: 5.2079. Min Act: -0.1700

Data Index: 2376124 (Open Web Text)

Max Activating Token Index: 879