Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2668 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.8459. Min Act: -0.1700

Data Index: 2550047 (Open Web Text)

Max Activating Token Index: 638

Text #1

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.8499. Min Act: -0.1700

Data Index: 5367542 (Open Web Text)

Max Activating Token Index: 78

Text #2

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.8413. Min Act: -0.1700

Data Index: 4774136 (Open Web Text)

Max Activating Token Index: 222

Text #3

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7998. Min Act: -0.1700

Data Index: 814362 (Open Web Text)

Max Activating Token Index: 402

Text #4

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7883. Min Act: -0.1700

Data Index: 4112102 (Open Web Text)

Max Activating Token Index: 604

Text #5

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7946. Min Act: -0.1700

Data Index: 5367543 (Open Web Text)

Max Activating Token Index: 952

Text #6

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7787. Min Act: -0.1700

Data Index: 5670624 (Open Web Text)

Max Activating Token Index: 840

Text #7

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7721. Min Act: -0.1700

Data Index: 7399046 (Open Web Text)

Max Activating Token Index: 550

Text #8

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7499. Min Act: -0.1700

Data Index: 3190004 (Open Web Text)

Max Activating Token Index: 47

Text #9

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7289. Min Act: -0.1695

Data Index: 5340351 (Open Web Text)

Max Activating Token Index: 170

Text #10

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7380. Min Act: -0.1700

Data Index: 1003593 (Open Web Text)

Max Activating Token Index: 463

Text #11

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7103. Min Act: -0.1700

Data Index: 2299347 (Open Web Text)

Max Activating Token Index: 59

Text #12

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7159. Min Act: -0.1700

Data Index: 7535444 (Open Web Text)

Max Activating Token Index: 695

Text #13

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7136. Min Act: -0.1700

Data Index: 5289041 (Open Web Text)

Max Activating Token Index: 585

Text #14

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7146. Min Act: -0.1700

Data Index: 1450091 (Open Web Text)

Max Activating Token Index: 765

Text #15

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7044. Min Act: -0.1700

Data Index: 5198017 (Open Web Text)

Max Activating Token Index: 688

Text #16

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.7149. Min Act: -0.1700

Data Index: 7031413 (Open Web Text)

Max Activating Token Index: 90

Text #17

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.6937. Min Act: -0.1700

Data Index: 339696 (Open Web Text)

Max Activating Token Index: 605

Text #18

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.6821. Min Act: -0.1700

Data Index: 2550048 (Open Web Text)

Max Activating Token Index: 130

Text #19

Max Range: 1.8499. Min Range: -1.8499

Max Act: 1.6915. Min Act: -0.1700

Data Index: 3777846 (Open Web Text)

Max Activating Token Index: 455