Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1721 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 6.2052. Min Range: -6.2052

Max Act: 6.2052. Min Act: -0.1700

Data Index: 8506636 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 6.2052. Min Range: -6.2052

Max Act: 6.1236. Min Act: -0.1700

Data Index: 6797542 (Open Web Text)

Max Activating Token Index: 7

Text #2

Max Range: 6.2052. Min Range: -6.2052

Max Act: 6.0668. Min Act: -0.1700

Data Index: 8698591 (Open Web Text)

Max Activating Token Index: 4

Text #3

Max Range: 6.2052. Min Range: -6.2052

Max Act: 6.0386. Min Act: -0.1700

Data Index: 3929455 (Open Web Text)

Max Activating Token Index: 3

Text #4

Max Range: 6.2052. Min Range: -6.2052

Max Act: 6.0429. Min Act: -0.1700

Data Index: 1967078 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 6.2052. Min Range: -6.2052

Max Act: 6.0011. Min Act: -0.1700

Data Index: 380514 (Open Web Text)

Max Activating Token Index: 4

Text #6

Max Range: 6.2052. Min Range: -6.2052

Max Act: 6.0269. Min Act: -0.1700

Data Index: 6706512 (Open Web Text)

Max Activating Token Index: 8

Text #7

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9682. Min Act: -0.1700

Data Index: 8046373 (Open Web Text)

Max Activating Token Index: 4

Text #8

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9833. Min Act: -0.1700

Data Index: 6675111 (Open Web Text)

Max Activating Token Index: 4

Text #9

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9430. Min Act: -0.1700

Data Index: 2658764 (Open Web Text)

Max Activating Token Index: 6

Text #10

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9324. Min Act: -0.1700

Data Index: 6584623 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9634. Min Act: -0.1700

Data Index: 1706263 (Open Web Text)

Max Activating Token Index: 4

Text #12

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9358. Min Act: -0.1700

Data Index: 831859 (Open Web Text)

Max Activating Token Index: 9

Text #13

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9360. Min Act: -0.1700

Data Index: 7067347 (Open Web Text)

Max Activating Token Index: 6

Text #14

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9469. Min Act: -0.1700

Data Index: 1338702 (Open Web Text)

Max Activating Token Index: 5

Text #15

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9220. Min Act: -0.1700

Data Index: 2071734 (Open Web Text)

Max Activating Token Index: 6

Text #16

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9237. Min Act: -0.1700

Data Index: 4258451 (Open Web Text)

Max Activating Token Index: 6

Text #17

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9012. Min Act: -0.1700

Data Index: 2102057 (Open Web Text)

Max Activating Token Index: 7

Text #18

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9190. Min Act: -0.1700

Data Index: 4979836 (Open Web Text)

Max Activating Token Index: 3

Text #19

Max Range: 6.2052. Min Range: -6.2052

Max Act: 5.9143. Min Act: -0.1700

Data Index: 1303993 (Open Web Text)

Max Activating Token Index: 5