Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1269 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2900. Min Act: -0.1700

Data Index: 1352173 (Open Web Text)

Max Activating Token Index: 3

Text #1

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2651. Min Act: -0.1700

Data Index: 6307798 (Open Web Text)

Max Activating Token Index: 3

Text #2

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2646. Min Act: -0.1700

Data Index: 7580744 (Open Web Text)

Max Activating Token Index: 3

Text #3

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2465. Min Act: -0.1700

Data Index: 2898933 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2449. Min Act: -0.1700

Data Index: 1939698 (Open Web Text)

Max Activating Token Index: 3

Text #5

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2476. Min Act: -0.1700

Data Index: 6112460 (Open Web Text)

Max Activating Token Index: 3

Text #6

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2433. Min Act: -0.1700

Data Index: 576758 (Open Web Text)

Max Activating Token Index: 3

Text #7

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2510. Min Act: -0.1700

Data Index: 7841439 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2528. Min Act: -0.1700

Data Index: 1020156 (Open Web Text)

Max Activating Token Index: 2

Text #9

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2365. Min Act: -0.1700

Data Index: 4518901 (Open Web Text)

Max Activating Token Index: 3

Text #10

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2410. Min Act: -0.1700

Data Index: 1817884 (Open Web Text)

Max Activating Token Index: 3

Text #11

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2435. Min Act: -0.1700

Data Index: 1712742 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2327. Min Act: -0.1700

Data Index: 5136854 (Open Web Text)

Max Activating Token Index: 3

Text #13

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2300. Min Act: -0.1700

Data Index: 7549705 (Open Web Text)

Max Activating Token Index: 3

Text #14

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2278. Min Act: -0.1700

Data Index: 7542152 (Open Web Text)

Max Activating Token Index: 2

Text #15

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2375. Min Act: -0.1700

Data Index: 7193410 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2314. Min Act: -0.1700

Data Index: 6521962 (Open Web Text)

Max Activating Token Index: 3

Text #17

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2278. Min Act: -0.1700

Data Index: 6458268 (Open Web Text)

Max Activating Token Index: 2

Text #18

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2395. Min Act: -0.1700

Data Index: 1014941 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 3.2900. Min Range: -3.2900

Max Act: 3.2367. Min Act: -0.1700

Data Index: 204069 (Open Web Text)

Max Activating Token Index: 2