Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2402 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.4365. Min Act: -0.1700

Data Index: 3049295 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3878. Min Act: -0.1700

Data Index: 4553730 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3957. Min Act: -0.1700

Data Index: 7940667 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3506. Min Act: -0.1700

Data Index: 7166018 (Open Web Text)

Max Activating Token Index: 7

Text #4

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3515. Min Act: -0.1700

Data Index: 3958106 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3702. Min Act: -0.1700

Data Index: 4128337 (Open Web Text)

Max Activating Token Index: 6

Text #6

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3808. Min Act: -0.1700

Data Index: 94893 (Open Web Text)

Max Activating Token Index: 7

Text #7

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3491. Min Act: -0.1700

Data Index: 1437788 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3254. Min Act: -0.1700

Data Index: 1234105 (Open Web Text)

Max Activating Token Index: 7

Text #9

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3333. Min Act: -0.1700

Data Index: 4423485 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3396. Min Act: -0.1700

Data Index: 1512601 (Open Web Text)

Max Activating Token Index: 5

Text #11

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3070. Min Act: -0.1700

Data Index: 3145629 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3059. Min Act: -0.1700

Data Index: 3435331 (Open Web Text)

Max Activating Token Index: 7

Text #13

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.2958. Min Act: -0.1700

Data Index: 2274751 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3020. Min Act: -0.1700

Data Index: 2957951 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3166. Min Act: -0.1700

Data Index: 1046201 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3017. Min Act: -0.1700

Data Index: 3752760 (Open Web Text)

Max Activating Token Index: 5

Text #17

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.2807. Min Act: -0.1700

Data Index: 5348224 (Open Web Text)

Max Activating Token Index: 5

Text #18

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.2868. Min Act: -0.1700

Data Index: 6080481 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 5.4365. Min Range: -5.4365

Max Act: 5.3040. Min Act: -0.1700

Data Index: 6909705 (Open Web Text)

Max Activating Token Index: 4