Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1063 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1163. Min Range: -4.1163

Max Act: 4.1163. Min Act: -0.1700

Data Index: 5735173 (Open Web Text)

Max Activating Token Index: 276

Text #1

Max Range: 4.1163. Min Range: -4.1163

Max Act: 4.0578. Min Act: -0.1700

Data Index: 6364120 (Open Web Text)

Max Activating Token Index: 110

Text #2

Max Range: 4.1163. Min Range: -4.1163

Max Act: 4.0375. Min Act: -0.1700

Data Index: 6741693 (Open Web Text)

Max Activating Token Index: 506

Text #3

Max Range: 4.1163. Min Range: -4.1163

Max Act: 4.0103. Min Act: -0.1700

Data Index: 364729 (Open Web Text)

Max Activating Token Index: 45

Text #4

Max Range: 4.1163. Min Range: -4.1163

Max Act: 4.0206. Min Act: -0.1700

Data Index: 1495102 (Open Web Text)

Max Activating Token Index: 231

Text #5

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.9727. Min Act: -0.1700

Data Index: 4729394 (Open Web Text)

Max Activating Token Index: 475

Text #6

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.9302. Min Act: -0.1700

Data Index: 1437346 (Open Web Text)

Max Activating Token Index: 871

Text #7

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.8954. Min Act: -0.1700

Data Index: 2101847 (Open Web Text)

Max Activating Token Index: 592

Text #8

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.9292. Min Act: -0.1700

Data Index: 6413138 (Open Web Text)

Max Activating Token Index: 182

Text #9

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.9219. Min Act: -0.1700

Data Index: 4752652 (Open Web Text)

Max Activating Token Index: 878

Text #10

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.9687. Min Act: -0.1700

Data Index: 6033038 (Open Web Text)

Max Activating Token Index: 661

Text #11

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.9146. Min Act: -0.1700

Data Index: 542150 (Open Web Text)

Max Activating Token Index: 463

Text #12

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.9493. Min Act: -0.1700

Data Index: 2429412 (Open Web Text)

Max Activating Token Index: 1003

Text #13

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.9128. Min Act: -0.1700

Data Index: 8341774 (Open Web Text)

Max Activating Token Index: 567

Text #14

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.9385. Min Act: -0.1700

Data Index: 1754111 (Open Web Text)

Max Activating Token Index: 816

Text #15

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.9234. Min Act: -0.1700

Data Index: 7445620 (Open Web Text)

Max Activating Token Index: 1014

Text #16

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.8815. Min Act: -0.1700

Data Index: 484006 (Open Web Text)

Max Activating Token Index: 736

Text #17

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.8734. Min Act: -0.1700

Data Index: 2319268 (Open Web Text)

Max Activating Token Index: 356

Text #18

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.8780. Min Act: -0.1700

Data Index: 4857600 (Open Web Text)

Max Activating Token Index: 269

Text #19

Max Range: 4.1163. Min Range: -4.1163

Max Act: 3.8497. Min Act: -0.1700

Data Index: 5013407 (Open Web Text)

Max Activating Token Index: 336