Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4618 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.1286. Min Range: -1.1286

Max Act: 1.1286. Min Act: -0.0000

Data Index: 99569 (The Pile)

Max Activating Token Index: 167

Text #1

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.6862. Min Act: -0.0000

Data Index: 404233 (The Pile)

Max Activating Token Index: 183

Text #2

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.6671. Min Act: -0.0000

Data Index: 837505 (The Pile)

Max Activating Token Index: 750

Text #3

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.6084. Min Act: -0.0000

Data Index: 238018 (The Pile)

Max Activating Token Index: 837

Text #4

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.6941. Min Act: -0.0000

Data Index: 1760598 (The Pile)

Max Activating Token Index: 62

Text #5

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.6890. Min Act: -0.0000

Data Index: 1714728 (The Pile)

Max Activating Token Index: 772

Text #6

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.6462. Min Act: -0.0000

Data Index: 309923 (The Pile)

Max Activating Token Index: 27

Text #7

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.6041. Min Act: -0.0000

Data Index: 1639917 (The Pile)

Max Activating Token Index: 221

Text #8

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.5705. Min Act: -0.0000

Data Index: 1533644 (The Pile)

Max Activating Token Index: 401

Text #9

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.6441. Min Act: -0.0000

Data Index: 264159 (The Pile)

Max Activating Token Index: 812

Text #10

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.5125. Min Act: -0.0000

Data Index: 660205 (The Pile)

Max Activating Token Index: 720

Text #11

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.5113. Min Act: -0.0000

Data Index: 1901109 (The Pile)

Max Activating Token Index: 788

Text #12

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.5408. Min Act: -0.0000

Data Index: 1212541 (The Pile)

Max Activating Token Index: 231

Text #13

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.5434. Min Act: -0.0000

Data Index: 917685 (The Pile)

Max Activating Token Index: 907

Text #14

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.5307. Min Act: -0.0000

Data Index: 139094 (The Pile)

Max Activating Token Index: 769

Text #15

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.5133. Min Act: -0.0000

Data Index: 675642 (The Pile)

Max Activating Token Index: 154

Text #16

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.5473. Min Act: -0.0000

Data Index: 1679596 (The Pile)

Max Activating Token Index: 691

Text #17

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.5021. Min Act: -0.0000

Data Index: 864505 (The Pile)

Max Activating Token Index: 578

Text #18

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.5535. Min Act: -0.0000

Data Index: 598467 (The Pile)

Max Activating Token Index: 305

Text #19

Max Range: 1.1286. Min Range: -1.1286

Max Act: 0.4971. Min Act: -0.0000

Data Index: 1339334 (The Pile)

Max Activating Token Index: 762