Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2869 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2759. Min Range: -5.2759

Max Act: 5.2759. Min Act: -0.1700

Data Index: 7412936 (Open Web Text)

Max Activating Token Index: 60

Text #1

Max Range: 5.2759. Min Range: -5.2759

Max Act: 5.1839. Min Act: -0.1700

Data Index: 5543102 (Open Web Text)

Max Activating Token Index: 412

Text #2

Max Range: 5.2759. Min Range: -5.2759

Max Act: 5.0975. Min Act: -0.1700

Data Index: 1888969 (Open Web Text)

Max Activating Token Index: 587

Text #3

Max Range: 5.2759. Min Range: -5.2759

Max Act: 5.1609. Min Act: -0.1700

Data Index: 3657511 (Open Web Text)

Max Activating Token Index: 417

Text #4

Max Range: 5.2759. Min Range: -5.2759

Max Act: 5.0669. Min Act: -0.1700

Data Index: 660837 (Open Web Text)

Max Activating Token Index: 814

Text #5

Max Range: 5.2759. Min Range: -5.2759

Max Act: 5.0298. Min Act: -0.1700

Data Index: 6355314 (Open Web Text)

Max Activating Token Index: 244

Text #6

Max Range: 5.2759. Min Range: -5.2759

Max Act: 5.0219. Min Act: -0.1700

Data Index: 321701 (Open Web Text)

Max Activating Token Index: 436

Text #7

Max Range: 5.2759. Min Range: -5.2759

Max Act: 5.0183. Min Act: -0.1700

Data Index: 888204 (Open Web Text)

Max Activating Token Index: 301

Text #8

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.9679. Min Act: -0.1700

Data Index: 6200794 (Open Web Text)

Max Activating Token Index: 913

Text #9

Max Range: 5.2759. Min Range: -5.2759

Max Act: 5.0240. Min Act: -0.1700

Data Index: 4210228 (Open Web Text)

Max Activating Token Index: 847

Text #10

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.9181. Min Act: -0.1700

Data Index: 5726605 (Open Web Text)

Max Activating Token Index: 809

Text #11

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.9086. Min Act: -0.1700

Data Index: 420225 (Open Web Text)

Max Activating Token Index: 252

Text #12

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.9378. Min Act: -0.1700

Data Index: 1356992 (Open Web Text)

Max Activating Token Index: 439

Text #13

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.9220. Min Act: -0.1700

Data Index: 4752463 (Open Web Text)

Max Activating Token Index: 291

Text #14

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.9292. Min Act: -0.1700

Data Index: 4136257 (Open Web Text)

Max Activating Token Index: 86

Text #15

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.9461. Min Act: -0.1700

Data Index: 5565649 (Open Web Text)

Max Activating Token Index: 169

Text #16

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.8753. Min Act: -0.1700

Data Index: 276306 (Open Web Text)

Max Activating Token Index: 208

Text #17

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.8992. Min Act: -0.1700

Data Index: 8721298 (Open Web Text)

Max Activating Token Index: 116

Text #18

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.8541. Min Act: -0.1696

Data Index: 5963691 (Open Web Text)

Max Activating Token Index: 234

Text #19

Max Range: 5.2759. Min Range: -5.2759

Max Act: 4.8816. Min Act: -0.1700

Data Index: 2299403 (Open Web Text)

Max Activating Token Index: 407