Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1775 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.0093. Min Range: -5.0093

Max Act: 5.0093. Min Act: -0.1700

Data Index: 7720855 (Open Web Text)

Max Activating Token Index: 19

Text #1

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.9411. Min Act: -0.1700

Data Index: 750234 (Open Web Text)

Max Activating Token Index: 148

Text #2

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.8392. Min Act: -0.1700

Data Index: 8488078 (Open Web Text)

Max Activating Token Index: 90

Text #3

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.8336. Min Act: -0.1700

Data Index: 1959608 (Open Web Text)

Max Activating Token Index: 590

Text #4

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.8434. Min Act: -0.1700

Data Index: 8674772 (Open Web Text)

Max Activating Token Index: 81

Text #5

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.8409. Min Act: -0.1700

Data Index: 3552996 (Open Web Text)

Max Activating Token Index: 36

Text #6

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.8400. Min Act: -0.1700

Data Index: 5687329 (Open Web Text)

Max Activating Token Index: 98

Text #7

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.7956. Min Act: -0.1700

Data Index: 2425071 (Open Web Text)

Max Activating Token Index: 220

Text #8

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.8155. Min Act: -0.1700

Data Index: 4185188 (Open Web Text)

Max Activating Token Index: 243

Text #9

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.7777. Min Act: -0.1700

Data Index: 1843883 (Open Web Text)

Max Activating Token Index: 1007

Text #10

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.8022. Min Act: -0.1700

Data Index: 3133656 (Open Web Text)

Max Activating Token Index: 90

Text #11

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.7943. Min Act: -0.1700

Data Index: 5569897 (Open Web Text)

Max Activating Token Index: 34

Text #12

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.8186. Min Act: -0.1700

Data Index: 3861464 (Open Web Text)

Max Activating Token Index: 498

Text #13

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.7798. Min Act: -0.1700

Data Index: 5834130 (Open Web Text)

Max Activating Token Index: 134

Text #14

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.7707. Min Act: -0.1700

Data Index: 6835862 (Open Web Text)

Max Activating Token Index: 204

Text #15

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.7010. Min Act: -0.1700

Data Index: 7769099 (Open Web Text)

Max Activating Token Index: 371

Text #16

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.7419. Min Act: -0.1700

Data Index: 5511481 (Open Web Text)

Max Activating Token Index: 216

Text #17

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.7371. Min Act: -0.1700

Data Index: 2739807 (Open Web Text)

Max Activating Token Index: 367

Text #18

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.7568. Min Act: -0.1691

Data Index: 876063 (Open Web Text)

Max Activating Token Index: 64

Text #19

Max Range: 5.0093. Min Range: -5.0093

Max Act: 4.7121. Min Act: -0.1700

Data Index: 5569896 (Open Web Text)

Max Activating Token Index: 908