Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1641 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.7694. Min Act: -0.1700

Data Index: 4713355 (Open Web Text)

Max Activating Token Index: 48

Text #1

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.7175. Min Act: -0.1700

Data Index: 8314115 (Open Web Text)

Max Activating Token Index: 43

Text #2

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.7103. Min Act: -0.1699

Data Index: 8527425 (Open Web Text)

Max Activating Token Index: 40

Text #3

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.7101. Min Act: -0.1700

Data Index: 6063782 (Open Web Text)

Max Activating Token Index: 42

Text #4

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.7050. Min Act: -0.1700

Data Index: 7835185 (Open Web Text)

Max Activating Token Index: 44

Text #5

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6965. Min Act: -0.1700

Data Index: 7334397 (Open Web Text)

Max Activating Token Index: 53

Text #6

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6926. Min Act: -0.1700

Data Index: 5344378 (Open Web Text)

Max Activating Token Index: 56

Text #7

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6885. Min Act: -0.1700

Data Index: 2511733 (Open Web Text)

Max Activating Token Index: 40

Text #8

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6785. Min Act: -0.1700

Data Index: 7610199 (Open Web Text)

Max Activating Token Index: 44

Text #9

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6845. Min Act: -0.1700

Data Index: 6343560 (Open Web Text)

Max Activating Token Index: 39

Text #10

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6800. Min Act: -0.1700

Data Index: 5774274 (Open Web Text)

Max Activating Token Index: 57

Text #11

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6772. Min Act: -0.1700

Data Index: 2581088 (Open Web Text)

Max Activating Token Index: 47

Text #12

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6771. Min Act: -0.1700

Data Index: 631233 (Open Web Text)

Max Activating Token Index: 52

Text #13

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6794. Min Act: -0.1700

Data Index: 5631700 (Open Web Text)

Max Activating Token Index: 58

Text #14

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6795. Min Act: -0.1700

Data Index: 5405303 (Open Web Text)

Max Activating Token Index: 40

Text #15

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6746. Min Act: -0.1700

Data Index: 1286525 (Open Web Text)

Max Activating Token Index: 44

Text #16

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6878. Min Act: -0.1700

Data Index: 4784186 (Open Web Text)

Max Activating Token Index: 47

Text #17

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6781. Min Act: -0.1700

Data Index: 3730483 (Open Web Text)

Max Activating Token Index: 46

Text #18

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6763. Min Act: -0.1700

Data Index: 516365 (Open Web Text)

Max Activating Token Index: 44

Text #19

Max Range: 1.7694. Min Range: -1.7694

Max Act: 1.6720. Min Act: -0.1700

Data Index: 1704 (Open Web Text)

Max Activating Token Index: 54