Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2279 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.4880. Min Act: -0.1700

Data Index: 2622681 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.4222. Min Act: -0.1700

Data Index: 2796574 (Open Web Text)

Max Activating Token Index: 5

Text #2

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.3193. Min Act: -0.1700

Data Index: 5372827 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.2802. Min Act: -0.1700

Data Index: 4118753 (Open Web Text)

Max Activating Token Index: 7

Text #4

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.2673. Min Act: -0.1700

Data Index: 8764110 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.2365. Min Act: -0.1700

Data Index: 7633757 (Open Web Text)

Max Activating Token Index: 5

Text #6

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.2434. Min Act: -0.1700

Data Index: 5898245 (Open Web Text)

Max Activating Token Index: 5

Text #7

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.2338. Min Act: -0.1700

Data Index: 3072341 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1996. Min Act: -0.1700

Data Index: 2085574 (Open Web Text)

Max Activating Token Index: 8

Text #9

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1920. Min Act: -0.1700

Data Index: 3807437 (Open Web Text)

Max Activating Token Index: 8

Text #10

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1879. Min Act: -0.1700

Data Index: 1808540 (Open Web Text)

Max Activating Token Index: 3

Text #11

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.2009. Min Act: -0.1700

Data Index: 8000744 (Open Web Text)

Max Activating Token Index: 7

Text #12

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1766. Min Act: -0.1700

Data Index: 7646862 (Open Web Text)

Max Activating Token Index: 6

Text #13

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1853. Min Act: -0.1700

Data Index: 4001242 (Open Web Text)

Max Activating Token Index: 7

Text #14

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1712. Min Act: -0.1700

Data Index: 435996 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1752. Min Act: -0.1700

Data Index: 1835536 (Open Web Text)

Max Activating Token Index: 6

Text #16

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1733. Min Act: -0.1700

Data Index: 239162 (Open Web Text)

Max Activating Token Index: 8

Text #17

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1664. Min Act: -0.1700

Data Index: 3548415 (Open Web Text)

Max Activating Token Index: 6

Text #18

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1639. Min Act: -0.1700

Data Index: 1701656 (Open Web Text)

Max Activating Token Index: 2

Text #19

Max Range: 4.4880. Min Range: -4.4880

Max Act: 4.1540. Min Act: -0.1700

Data Index: 1381106 (Open Web Text)

Max Activating Token Index: 3