Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2366 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.2088. Min Act: -0.1700

Data Index: 6414097 (Open Web Text)

Max Activating Token Index: 751

Text #1

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.1466. Min Act: -0.1700

Data Index: 8664655 (Open Web Text)

Max Activating Token Index: 409

Text #2

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.1026. Min Act: -0.1700

Data Index: 3113890 (Open Web Text)

Max Activating Token Index: 426

Text #3

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.0934. Min Act: -0.1700

Data Index: 1131347 (Open Web Text)

Max Activating Token Index: 402

Text #4

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.0656. Min Act: -0.1700

Data Index: 4441857 (Open Web Text)

Max Activating Token Index: 241

Text #5

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.0498. Min Act: -0.1700

Data Index: 629788 (Open Web Text)

Max Activating Token Index: 343

Text #6

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.0520. Min Act: -0.1700

Data Index: 1078889 (Open Web Text)

Max Activating Token Index: 581

Text #7

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.0541. Min Act: -0.1700

Data Index: 6766120 (Open Web Text)

Max Activating Token Index: 975

Text #8

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.0059. Min Act: -0.1700

Data Index: 6161757 (Open Web Text)

Max Activating Token Index: 46

Text #9

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.0048. Min Act: -0.1700

Data Index: 163424 (Open Web Text)

Max Activating Token Index: 845

Text #10

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.0182. Min Act: -0.1700

Data Index: 483261 (Open Web Text)

Max Activating Token Index: 223

Text #11

Max Range: 3.2088. Min Range: -3.2088

Max Act: 2.9744. Min Act: -0.1700

Data Index: 1570128 (Open Web Text)

Max Activating Token Index: 105

Text #12

Max Range: 3.2088. Min Range: -3.2088

Max Act: 2.9707. Min Act: -0.1700

Data Index: 8031294 (Open Web Text)

Max Activating Token Index: 267

Text #13

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.0151. Min Act: -0.1700

Data Index: 3959610 (Open Web Text)

Max Activating Token Index: 815

Text #14

Max Range: 3.2088. Min Range: -3.2088

Max Act: 3.0131. Min Act: -0.1700

Data Index: 567467 (Open Web Text)

Max Activating Token Index: 911

Text #15

Max Range: 3.2088. Min Range: -3.2088

Max Act: 2.9821. Min Act: -0.1700

Data Index: 5257437 (Open Web Text)

Max Activating Token Index: 462

Text #16

Max Range: 3.2088. Min Range: -3.2088

Max Act: 2.9495. Min Act: -0.1700

Data Index: 4554841 (Open Web Text)

Max Activating Token Index: 51

Text #17

Max Range: 3.2088. Min Range: -3.2088

Max Act: 2.9456. Min Act: -0.1700

Data Index: 3004118 (Open Web Text)

Max Activating Token Index: 383

Text #18

Max Range: 3.2088. Min Range: -3.2088

Max Act: 2.9657. Min Act: -0.1700

Data Index: 5746104 (Open Web Text)

Max Activating Token Index: 702

Text #19

Max Range: 3.2088. Min Range: -3.2088

Max Act: 2.9621. Min Act: -0.1700

Data Index: 2602069 (Open Web Text)

Max Activating Token Index: 919