Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2287 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.4795. Min Act: -0.1700

Data Index: 804771 (Open Web Text)

Max Activating Token Index: 335

Text #1

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.4045. Min Act: -0.1700

Data Index: 6528333 (Open Web Text)

Max Activating Token Index: 423

Text #2

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.3785. Min Act: -0.1700

Data Index: 4894204 (Open Web Text)

Max Activating Token Index: 740

Text #3

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.3366. Min Act: -0.1700

Data Index: 6345581 (Open Web Text)

Max Activating Token Index: 599

Text #4

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.3442. Min Act: -0.1700

Data Index: 6315870 (Open Web Text)

Max Activating Token Index: 205

Text #5

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.3435. Min Act: -0.1700

Data Index: 3031168 (Open Web Text)

Max Activating Token Index: 57

Text #6

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.3457. Min Act: -0.1700

Data Index: 7493953 (Open Web Text)

Max Activating Token Index: 955

Text #7

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.3062. Min Act: -0.1700

Data Index: 4118990 (Open Web Text)

Max Activating Token Index: 209

Text #8

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.3178. Min Act: -0.1700

Data Index: 2102726 (Open Web Text)

Max Activating Token Index: 52

Text #9

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2837. Min Act: -0.1700

Data Index: 2901965 (Open Web Text)

Max Activating Token Index: 669

Text #10

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2943. Min Act: -0.1700

Data Index: 2863337 (Open Web Text)

Max Activating Token Index: 282

Text #11

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2925. Min Act: -0.1700

Data Index: 242412 (Open Web Text)

Max Activating Token Index: 117

Text #12

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2500. Min Act: -0.1700

Data Index: 6309934 (Open Web Text)

Max Activating Token Index: 516

Text #13

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2571. Min Act: -0.1700

Data Index: 4563182 (Open Web Text)

Max Activating Token Index: 505

Text #14

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2808. Min Act: -0.1700

Data Index: 4106914 (Open Web Text)

Max Activating Token Index: 730

Text #15

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2606. Min Act: -0.1700

Data Index: 5768426 (Open Web Text)

Max Activating Token Index: 390

Text #16

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2530. Min Act: -0.1700

Data Index: 5021353 (Open Web Text)

Max Activating Token Index: 76

Text #17

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2443. Min Act: -0.1700

Data Index: 8693 (Open Web Text)

Max Activating Token Index: 354

Text #18

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2454. Min Act: -0.1700

Data Index: 6602902 (Open Web Text)

Max Activating Token Index: 547

Text #19

Max Range: 5.4795. Min Range: -5.4795

Max Act: 5.2677. Min Act: -0.1700

Data Index: 7033569 (Open Web Text)

Max Activating Token Index: 529