Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 101 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.6556. Min Act: -0.1700

Data Index: 4963816 (Open Web Text)

Max Activating Token Index: 221

Text #1

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.7230. Min Act: -0.1700

Data Index: 5202543 (Open Web Text)

Max Activating Token Index: 619

Text #2

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.5745. Min Act: -0.1700

Data Index: 7872796 (Open Web Text)

Max Activating Token Index: 111

Text #3

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.5239. Min Act: -0.1700

Data Index: 4309393 (Open Web Text)

Max Activating Token Index: 587

Text #4

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.4457. Min Act: -0.1700

Data Index: 3453038 (Open Web Text)

Max Activating Token Index: 162

Text #5

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.3690. Min Act: -0.1700

Data Index: 3457670 (Open Web Text)

Max Activating Token Index: 744

Text #6

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.2603. Min Act: -0.1700

Data Index: 4810926 (Open Web Text)

Max Activating Token Index: 341

Text #7

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.2718. Min Act: -0.1700

Data Index: 7038007 (Open Web Text)

Max Activating Token Index: 321

Text #8

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.2780. Min Act: -0.1700

Data Index: 263182 (Open Web Text)

Max Activating Token Index: 706

Text #9

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.2507. Min Act: -0.1700

Data Index: 5677470 (Open Web Text)

Max Activating Token Index: 467

Text #10

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.2060. Min Act: -0.1700

Data Index: 4269240 (Open Web Text)

Max Activating Token Index: 997

Text #11

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.1980. Min Act: -0.1700

Data Index: 7750097 (Open Web Text)

Max Activating Token Index: 373

Text #12

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.2238. Min Act: -0.1700

Data Index: 3165773 (Open Web Text)

Max Activating Token Index: 894

Text #13

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.2150. Min Act: -0.1700

Data Index: 8730676 (Open Web Text)

Max Activating Token Index: 1020

Text #14

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.1773. Min Act: -0.1700

Data Index: 7217853 (Open Web Text)

Max Activating Token Index: 178

Text #15

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.1583. Min Act: -0.1700

Data Index: 2805027 (Open Web Text)

Max Activating Token Index: 267

Text #16

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.1821. Min Act: -0.1700

Data Index: 1032557 (Open Web Text)

Max Activating Token Index: 28

Text #17

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.1961. Min Act: -0.1700

Data Index: 559430 (Open Web Text)

Max Activating Token Index: 358

Text #18

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.2143. Min Act: -0.1700

Data Index: 8250093 (Open Web Text)

Max Activating Token Index: 825

Text #19

Max Range: 5.7230. Min Range: -5.7230

Max Act: 5.2078. Min Act: -0.1700

Data Index: 6844567 (Open Web Text)

Max Activating Token Index: 834