Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 943 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.4838. Min Range: -5.4838

Max Act: 5.4838. Min Act: -0.1700

Data Index: 2972769 (Open Web Text)

Max Activating Token Index: 505

Text #1

Max Range: 5.4838. Min Range: -5.4838

Max Act: 5.1789. Min Act: -0.1700

Data Index: 8002012 (Open Web Text)

Max Activating Token Index: 621

Text #2

Max Range: 5.4838. Min Range: -5.4838

Max Act: 5.1359. Min Act: -0.1700

Data Index: 4015224 (Open Web Text)

Max Activating Token Index: 494

Text #3

Max Range: 5.4838. Min Range: -5.4838

Max Act: 5.0991. Min Act: -0.1700

Data Index: 2610971 (Open Web Text)

Max Activating Token Index: 177

Text #4

Max Range: 5.4838. Min Range: -5.4838

Max Act: 5.0877. Min Act: -0.1700

Data Index: 2610970 (Open Web Text)

Max Activating Token Index: 371

Text #5

Max Range: 5.4838. Min Range: -5.4838

Max Act: 5.0774. Min Act: -0.1700

Data Index: 4620780 (Open Web Text)

Max Activating Token Index: 768

Text #6

Max Range: 5.4838. Min Range: -5.4838

Max Act: 5.0299. Min Act: -0.1700

Data Index: 2656306 (Open Web Text)

Max Activating Token Index: 311

Text #7

Max Range: 5.4838. Min Range: -5.4838

Max Act: 5.0887. Min Act: -0.1700

Data Index: 1751549 (Open Web Text)

Max Activating Token Index: 300

Text #8

Max Range: 5.4838. Min Range: -5.4838

Max Act: 5.0737. Min Act: -0.1700

Data Index: 1574932 (Open Web Text)

Max Activating Token Index: 302

Text #9

Max Range: 5.4838. Min Range: -5.4838

Max Act: 5.0320. Min Act: -0.1700

Data Index: 2043740 (Open Web Text)

Max Activating Token Index: 690

Text #10

Max Range: 5.4838. Min Range: -5.4838

Max Act: 4.9683. Min Act: -0.1700

Data Index: 997035 (Open Web Text)

Max Activating Token Index: 539

Text #11

Max Range: 5.4838. Min Range: -5.4838

Max Act: 4.9856. Min Act: -0.1700

Data Index: 2212447 (Open Web Text)

Max Activating Token Index: 751

Text #12

Max Range: 5.4838. Min Range: -5.4838

Max Act: 4.9758. Min Act: -0.1700

Data Index: 1742941 (Open Web Text)

Max Activating Token Index: 601

Text #13

Max Range: 5.4838. Min Range: -5.4838

Max Act: 4.9657. Min Act: -0.1700

Data Index: 4441283 (Open Web Text)

Max Activating Token Index: 235

Text #14

Max Range: 5.4838. Min Range: -5.4838

Max Act: 4.9751. Min Act: -0.1699

Data Index: 2525305 (Open Web Text)

Max Activating Token Index: 998

Text #15

Max Range: 5.4838. Min Range: -5.4838

Max Act: 4.9658. Min Act: -0.1700

Data Index: 108672 (Open Web Text)

Max Activating Token Index: 936

Text #16

Max Range: 5.4838. Min Range: -5.4838

Max Act: 4.9661. Min Act: -0.1700

Data Index: 2361712 (Open Web Text)

Max Activating Token Index: 214

Text #17

Max Range: 5.4838. Min Range: -5.4838

Max Act: 4.9174. Min Act: -0.1700

Data Index: 561013 (Open Web Text)

Max Activating Token Index: 426

Text #18

Max Range: 5.4838. Min Range: -5.4838

Max Act: 4.9346. Min Act: -0.1700

Data Index: 4801020 (Open Web Text)

Max Activating Token Index: 461

Text #19

Max Range: 5.4838. Min Range: -5.4838

Max Act: 4.9476. Min Act: -0.1700

Data Index: 4329471 (Open Web Text)

Max Activating Token Index: 508