Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1007 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.9503. Min Act: -0.1700

Data Index: 7712987 (Open Web Text)

Max Activating Token Index: 154

Text #1

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.9512. Min Act: -0.1700

Data Index: 2155567 (Open Web Text)

Max Activating Token Index: 187

Text #2

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.9418. Min Act: -0.1700

Data Index: 5531128 (Open Web Text)

Max Activating Token Index: 131

Text #3

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.9221. Min Act: -0.1700

Data Index: 3782781 (Open Web Text)

Max Activating Token Index: 46

Text #4

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.9204. Min Act: -0.1700

Data Index: 8068425 (Open Web Text)

Max Activating Token Index: 101

Text #5

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.9266. Min Act: -0.1700

Data Index: 7353325 (Open Web Text)

Max Activating Token Index: 63

Text #6

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.9141. Min Act: -0.1700

Data Index: 2013271 (Open Web Text)

Max Activating Token Index: 72

Text #7

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.9010. Min Act: -0.1700

Data Index: 3066787 (Open Web Text)

Max Activating Token Index: 489

Text #8

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8969. Min Act: -0.1700

Data Index: 1941089 (Open Web Text)

Max Activating Token Index: 712

Text #9

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.9034. Min Act: -0.1700

Data Index: 819095 (Open Web Text)

Max Activating Token Index: 880

Text #10

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8859. Min Act: -0.1700

Data Index: 2361766 (Open Web Text)

Max Activating Token Index: 958

Text #11

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8925. Min Act: -0.1700

Data Index: 3580037 (Open Web Text)

Max Activating Token Index: 449

Text #12

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8954. Min Act: -0.1700

Data Index: 3984094 (Open Web Text)

Max Activating Token Index: 708

Text #13

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8852. Min Act: -0.1700

Data Index: 7127770 (Open Web Text)

Max Activating Token Index: 355

Text #14

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8881. Min Act: -0.1700

Data Index: 4681170 (Open Web Text)

Max Activating Token Index: 650

Text #15

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8853. Min Act: -0.1700

Data Index: 1941086 (Open Web Text)

Max Activating Token Index: 275

Text #16

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8948. Min Act: -0.1700

Data Index: 6418342 (Open Web Text)

Max Activating Token Index: 134

Text #17

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8942. Min Act: -0.1700

Data Index: 4279934 (Open Web Text)

Max Activating Token Index: 103

Text #18

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8870. Min Act: -0.1700

Data Index: 7706714 (Open Web Text)

Max Activating Token Index: 185

Text #19

Max Range: 1.9512. Min Range: -1.9512

Max Act: 1.8836. Min Act: -0.1700

Data Index: 3131758 (Open Web Text)

Max Activating Token Index: 189