Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1731 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.4127. Min Act: -0.1700

Data Index: 1231665 (Open Web Text)

Max Activating Token Index: 4

Text #1

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.3663. Min Act: -0.1700

Data Index: 2378992 (Open Web Text)

Max Activating Token Index: 4

Text #2

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.3373. Min Act: -0.1700

Data Index: 2442829 (Open Web Text)

Max Activating Token Index: 6

Text #3

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.3237. Min Act: -0.1700

Data Index: 567474 (Open Web Text)

Max Activating Token Index: 5

Text #4

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.3173. Min Act: -0.1700

Data Index: 174158 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2873. Min Act: -0.1700

Data Index: 7329736 (Open Web Text)

Max Activating Token Index: 704

Text #6

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2975. Min Act: -0.1700

Data Index: 7954065 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2651. Min Act: -0.1700

Data Index: 6088279 (Open Web Text)

Max Activating Token Index: 3

Text #8

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2663. Min Act: -0.1700

Data Index: 3384291 (Open Web Text)

Max Activating Token Index: 6

Text #9

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2775. Min Act: -0.1700

Data Index: 4185100 (Open Web Text)

Max Activating Token Index: 7

Text #10

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2798. Min Act: -0.1700

Data Index: 520799 (Open Web Text)

Max Activating Token Index: 4

Text #11

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2641. Min Act: -0.1700

Data Index: 1551792 (Open Web Text)

Max Activating Token Index: 4

Text #12

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2444. Min Act: -0.1700

Data Index: 4704187 (Open Web Text)

Max Activating Token Index: 4

Text #13

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2456. Min Act: -0.1700

Data Index: 852902 (Open Web Text)

Max Activating Token Index: 8

Text #14

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2719. Min Act: -0.1700

Data Index: 4289690 (Open Web Text)

Max Activating Token Index: 4

Text #15

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2714. Min Act: -0.1700

Data Index: 2690756 (Open Web Text)

Max Activating Token Index: 3

Text #16

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2609. Min Act: -0.1700

Data Index: 920559 (Open Web Text)

Max Activating Token Index: 4

Text #17

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2349. Min Act: -0.1700

Data Index: 3924291 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2196. Min Act: -0.1700

Data Index: 1557679 (Open Web Text)

Max Activating Token Index: 6

Text #19

Max Range: 4.4127. Min Range: -4.4127

Max Act: 4.2307. Min Act: -0.1700

Data Index: 174005 (Open Web Text)

Max Activating Token Index: 4