Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 310 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.1549. Min Act: -0.1700

Data Index: 7269784 (Open Web Text)

Max Activating Token Index: 880

Text #1

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0847. Min Act: -0.1700

Data Index: 6924977 (Open Web Text)

Max Activating Token Index: 2

Text #2

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0847. Min Act: -0.1700

Data Index: 6182329 (Open Web Text)

Max Activating Token Index: 2

Text #3

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0717. Min Act: -0.1700

Data Index: 2418023 (Open Web Text)

Max Activating Token Index: 928

Text #4

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0080. Min Act: -0.1700

Data Index: 2639798 (Open Web Text)

Max Activating Token Index: 762

Text #5

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0745. Min Act: -0.1700

Data Index: 461045 (Open Web Text)

Max Activating Token Index: 658

Text #6

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0151. Min Act: -0.1700

Data Index: 7316806 (Open Web Text)

Max Activating Token Index: 881

Text #7

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0705. Min Act: -0.1700

Data Index: 5836324 (Open Web Text)

Max Activating Token Index: 830

Text #8

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0493. Min Act: -0.1700

Data Index: 348454 (Open Web Text)

Max Activating Token Index: 682

Text #9

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0128. Min Act: -0.1700

Data Index: 445763 (Open Web Text)

Max Activating Token Index: 4

Text #10

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0224. Min Act: -0.1700

Data Index: 8544649 (Open Web Text)

Max Activating Token Index: 2

Text #11

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0211. Min Act: -0.1700

Data Index: 7898231 (Open Web Text)

Max Activating Token Index: 3

Text #12

Max Range: 3.1549. Min Range: -3.1549

Max Act: 2.9750. Min Act: -0.1700

Data Index: 5771602 (Open Web Text)

Max Activating Token Index: 657

Text #13

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0198. Min Act: -0.1700

Data Index: 450358 (Open Web Text)

Max Activating Token Index: 925

Text #14

Max Range: 3.1549. Min Range: -3.1549

Max Act: 2.9860. Min Act: -0.1700

Data Index: 7543113 (Open Web Text)

Max Activating Token Index: 886

Text #15

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0171. Min Act: -0.1700

Data Index: 2996075 (Open Web Text)

Max Activating Token Index: 811

Text #16

Max Range: 3.1549. Min Range: -3.1549

Max Act: 2.9807. Min Act: -0.1700

Data Index: 550048 (Open Web Text)

Max Activating Token Index: 917

Text #17

Max Range: 3.1549. Min Range: -3.1549

Max Act: 2.9590. Min Act: -0.1700

Data Index: 1285509 (Open Web Text)

Max Activating Token Index: 758

Text #18

Max Range: 3.1549. Min Range: -3.1549

Max Act: 2.9803. Min Act: -0.1700

Data Index: 3000710 (Open Web Text)

Max Activating Token Index: 17

Text #19

Max Range: 3.1549. Min Range: -3.1549

Max Act: 3.0272. Min Act: -0.1700

Data Index: 8652010 (Open Web Text)

Max Activating Token Index: 743